[SOLVED] Backpropagation RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

I am using LightCNN to extract a face features vector out of an image generated from a GAN. I want to compute the loss between the generated image from the GAN and the original face before going inside the GAN. Below is an example of the code I am using:

import torch
from util.util import get_image
import torchvision.transforms as transforms
from LightCNN.light_cnn import LightCNN_29Layers_v2

face_model = torch.nn.DataParallel(LightCNN_29Layers_v2(num_classes=80013)).cuda()
checkpoint = torch.load('LightCNN_29Layers_V2_checkpoint.pth')
face_model.load_state_dict(checkpoint['state_dict'])
face_model.eval()

transform = transforms.Compose([transforms.ToTensor()])
																	   #get_image returns an image as if it was read from disk, check https://github.com/AlfredXiangWu/LightCNN/blob/master/extract_features.py#L76
tmp_rec_A=transform(get_image(rec_A.detach())).unsqueeze(0)   #rec_A is the output generated from a GAN after giving it an input image A
tmp_real_A=transform(get_image(real_A)).unsqueeze(0)          #real_A is the original image A                  
tmp_rec_B=transform(get_image(rec_B.detach())).unsqueeze(0)   #rec_B is the output generated from a GAN after giving it an input image B
tmp_real_B=transform(get_image(real_B)).unsqueeze(0)          #real_B is the original image B                   


rec_A_feat =face_model(tmp_rec_A)[1]      #face feature vector of the output generated from a GAN after giving it an input image A
real_A_feat=face_model(tmp_real_A)[1] 	  #face feature vector of the original image A
rec_B_feat =face_model(tmp_rec_B)[1]      #face feature vector of the output generated from a GAN after giving it an input image B
real_B_feat=face_model(tmp_real_B)[1]     #face feature vector of the original image B

criterionFeatureMatch = torch.nn.MSELoss()

loss_face_feature_A = criterionFeatureMatch(rec_A_feat, real_A_feat)
loss_face_feature_B = criterionFeatureMatch(rec_B_feat, real_B_feat)           
loss_G = loss_face_feature_A + loss_face_feature_B
loss_G.backward() 						  #should backpropagate the gradients for the generator network

However, when the last line is executed I get the following error:

  File "anaconda3/envs/temp/lib/python3.6/site-packages/torch/tensor.py", line 102, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "anaconda3/envs/temp/lib/python3.6/site-packages/torch/autograd/__init__.py", line 90, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

My PyTorch version is 1.0.1 and Python version is 3.6.2

I do not know what is wrong with this code. Any help is appreciated. Thanks!

I don’t have a clear answer, as I don’t know this particular model, but let’s try to debug-by-questions.

  1. Have you tried to see which tensors require gradient and which don’t? In particular look at loss_face_feature_A and loss_face_feature_B.
  2. Moreover, if you need gradient I assume you want to optimize the model. In that case the call to face_model.eval() is not appropriate I think. When training, the model has to be in training mode.

Or am I missing something?

Thanks for your reply.

  1. I checked both loss_face_feature_A and loss_face_feature_B and they do not require gradients. I don’t know the reason for this.

  2. I don’t want to train face_model, I am just using it for inference to extract the face features before and after passing the face image into the GAN.

  1. Check the weights of your model, if they require grads with a for loop like this:
for param in model.parameters():
    print( param.require_grad )
  1. If you don’t want to train the model why are you backpropagating the loss? What do you use the computed gradient for?

I want to backpropagate the loss in the generator networks that produce rec_A and rec_B (the generator networks are not shown in the code for better readability).

The face_model is just used as a feature extractor and the loss is computed between the features of the face image produced from the generator, e.g. rec_A, and the features of the ground truth face image real_A.

I think the .detach() on rec_A and rec_B is wrong. However, even after removing them I still get the same error.

I found out the issue. I am using this Face Alignment model to align the face images and it turns out that internally it calls torch.set_grad_enabled(False) which apparently turns off the gradient computation globally in PyTorch.

To solve the issue I just call torch.set_grad_enabled(True) when training the GAN.

1 Like