Why no gradient calculated?

I defined 2 variables (D, ht) to received gradients. However after the loss function and backward() calculation, only D has gradients, ht does not have any gradients calculated. Can you please help me to understand why it is the case? thank you.

import torch.nn as nn
import torch
from torch.autograd import Variable, Function

x = Variable(torch.from_numpy(np.random.normal(0,1,(10,10))), requires_grad=False)  # original

# depends on size of the dictionary, number of atoms.
D = Variable(torch.from_numpy(np.random.normal(0,1,(500,10,10))), requires_grad=True)

# hx sparse representation
ht = Variable(torch.from_numpy(np.random.normal(0,1,(500,1,1))), requires_grad=True)
ld = 0.02
lr = 0.001 # learning rate

# optimizer
optimizer = torch.optim.SGD([ht,D], lr=lr, momentum=0.9)
optimizer.zero_grad()

loss_ht = 0.5*torch.norm((x-(D*ht).sum(dim=0)),p=2)**2
loss_ht.backward() # back propogation
optimizer.step() # update parameters

D has graidents, see below

D.grad.data

( 0 ,.,.) =
2.0307e+01 9.9208e+00 4.9194e+00 … -2.3104e+01 -2.6616e+01 -9.8070e-01
3.7742e+01 2.5255e+01 4.5286e+00 … -3.5697e+01 -1.1306e+01 1.9366e+01

ht does not have any gradients. but why? please help. see below.

ht.grad.data

AttributeError Traceback (most recent call last)
in ()
----> 1 ht.grad.data

AttributeError: ‘NoneType’ object has no attribute ‘data’

Hi,

Running the above code with master branch works as expected: both tensors have gradient. Which version of pytorch are you using?

1 Like

version ‘0.2.0_4’ is it too old?

@albanD
I just upgrade pytorch to 0.3.1. it gives the same result: no gradients for ht.

torch.__version__

‘0.3.1’

ht.grad.data

AttributeError Traceback (most recent call last)
in ()
----> 1 ht.grad.data

AttributeError: ‘NoneType’ object has no attribute ‘data’

I just ran your code. It works perfectly for me. ht.grad.data is a tensor of 500x1x1 values.

which version do you have?

I have PyTorch 0.3.1

1 Like

you are right. my bad. I have one line of code mess up ht. Thanks!