Getting gradient of two losses

I am trying to get the gradients of two losses in the following code snippet but all I get is None (AttributeError: ‘NoneType’ object has no attribute ‘data’)

img =
#img.requires_grad = True
input = model(img)
input_prediction = input.max(1, keepdim=True)[1]

btarget = torch.tensor(2).unsqueeze(0).to(device)
x_prime.requires_grad = True
x_prime_output = model(x_prime)
x_prime_pred =  x_prime_output.max(1, keepdim=True)[1]
l_target_loss = F.nll_loss(x_prime_output, btarget)
# model.zero_grad()
l_target_loss.backward(retain_graph = True)
target_grad =

l_argmax_loss = F.nll_loss(x_prime_output, input_prediction.squeeze(0))
l_argmax_grad =

What am i getting wrong, please?

@albanD , could you help sir

In general, you should never user .data.

Also the gradient computed by the backward pass is for the model parameters (generally Tensors you create with requires_grad=True). So here you will get gradients for x_prime for example. Not the loss.

Thank you for your reply. I added l_target_loss.retain_grad() before calling l_target_loss.backward(retain_graph= True) and it worked. I can now get values for target_grad and l_argmax_grad respectively. However, both values are thesame. They are both 1’s. I feel something isn’t right

@albanD , I feel that I’m doing something wrong. What I actually want is the gradient of the target_loss with respect to the input (x) and gradient of the l_argmax_loss with respect to the input (x). The input x is img in the code snippet above. How do I go about it please?

They are both 1’s.

That is expected as they contain d l_argmax_loss / d l_argmax_loss Which is obviously 1.

gradient of the l_argmax_loss with respect to the input (x)

To get this quantity, you should:

  • make sure x.requires_grad=True
  • compute l_argmax_loss in a differentiable manner
  • call l_argmax_loss.backward()
  • read the gradient at x.grad