I am trying to get the gradients of two losses in the following code snippet but all I get is None (AttributeError: ‘NoneType’ object has no attribute ‘data’)
img = img.to(device)
#img.requires_grad = True
input = model(img)
input_prediction = input.max(1, keepdim=True)
btarget = torch.tensor(2).unsqueeze(0).to(device)
x_prime.requires_grad = True
x_prime_output = model(x_prime)
x_prime_pred = x_prime_output.max(1, keepdim=True)
l_target_loss = F.nll_loss(x_prime_output, btarget)
l_target_loss.backward(retain_graph = True)
target_grad = l_target_loss.grad.data
l_argmax_loss = F.nll_loss(x_prime_output, input_prediction.squeeze(0))
l_argmax_grad = l_argmax_loss.grad.data
What am i getting wrong, please?
@albanD , could you help sir
In general, you should never user .data.
Also the gradient computed by the backward pass is for the model parameters (generally Tensors you create with requires_grad=True). So here you will get gradients for x_prime for example. Not the loss.
Thank you for your reply. I added
l_target_loss.retain_grad() before calling
l_target_loss.backward(retain_graph= True) and it worked. I can now get values for
l_argmax_grad respectively. However, both values are thesame. They are both 1’s. I feel something isn’t right
@albanD , I feel that I’m doing something wrong. What I actually want is the gradient of the
target_loss with respect to the input (x) and gradient of the
l_argmax_loss with respect to the input (x). The input
img in the code snippet above. How do I go about it please?
They are both 1’s.
That is expected as they contain
d l_argmax_loss / d l_argmax_loss Which is obviously 1.
gradient of the
l_argmax_loss with respect to the input (x)
To get this quantity, you should:
- make sure x.requires_grad=True
- compute l_argmax_loss in a differentiable manner
- call l_argmax_loss.backward()
- read the gradient at x.grad