Another "Expected Variable(torch.FloatTensor) for Variable(torch.cuda.FloatTensor" but on loss.backward()

This was the warning that came when I called backward function

File "/home/anurag/naman/pytorch3/lib/python3.6/site-packages/torch/autograd/", line 99, in backward
    variables, grad_variables, retain_graph)
RuntimeError: Expected object of type Variable[torch.FloatTensor] but found type Variable[torch.cuda.FloatTensor] for argument #1 'mat2'

As I know this happens when model or input are non cuda but since forward worked so this is new to me.
Can anyone help out?

Created a Variable inside my model with torch.zeros(…) without cuda, so one of the output was non cuda.
This code worked in pytorch0.4!