Hi Krish,
Thanks for your reply.
Yes, the problem has been slightly solved. Now the model is training but it is not achieving the desired results. This is not related to the model or learning.
As you suggested before, the error was in gradients as they were removing before back tracking. Now, after creating all the torch variable on cuda, the model has started training. Thanks for your help. 
-Zohaib