I defined 2 variables (D, ht) to received gradients. However after the loss function and backward() calculation, only D has gradients, ht does not have any gradients calculated. Can you please help me to understand why it is the case? thank you.
import torch.nn as nn
import torch
from torch.autograd import Variable, Function
x = Variable(torch.from_numpy(np.random.normal(0,1,(10,10))), requires_grad=False) # original
# depends on size of the dictionary, number of atoms.
D = Variable(torch.from_numpy(np.random.normal(0,1,(500,10,10))), requires_grad=True)
# hx sparse representation
ht = Variable(torch.from_numpy(np.random.normal(0,1,(500,1,1))), requires_grad=True)
ld = 0.02
lr = 0.001 # learning rate
# optimizer
optimizer = torch.optim.SGD([ht,D], lr=lr, momentum=0.9)
optimizer.zero_grad()
loss_ht = 0.5*torch.norm((x-(D*ht).sum(dim=0)),p=2)**2
loss_ht.backward() # back propogation
optimizer.step() # update parameters
D has graidents, see below
D.grad.data
( 0 ,.,.) =
2.0307e+01 9.9208e+00 4.9194e+00 … -2.3104e+01 -2.6616e+01 -9.8070e-01
3.7742e+01 2.5255e+01 4.5286e+00 … -3.5697e+01 -1.1306e+01 1.9366e+01
ht does not have any gradients. but why? please help. see below.
ht.grad.data
AttributeError Traceback (most recent call last)
in ()
----> 1 ht.grad.data
AttributeError: ‘NoneType’ object has no attribute ‘data’