Hello, everyone. I need to use the neural network in an unconventional way, in which I have to compute the gradient of the model output with respect to the input, but I always get a None.
My code is like this:
model = torch.load('totalmodel.pth')
model.eval()
x = torch.tensor([1.,2.,3.,4.],device=device,requires_grad=True)
y = model(x)
y.backward()
print(y)
print(x.grad)
If the gradients are unexpectedly None, you could try the following simple checks -
tensor.requires_grad == True
tensor.is_leaf == True, or tensor.grad_fn is None;
if it is not None it implies the tensor isn’t a leaf tensor and you might want to use retain_grad() on it.
autograd’s gradient computation is not disabled using
• torch.no_grad() context manager
• torch.autograd.set_grad_enabled(False)
You are not running any non-differentiable operation and/or breaking the computation graph.
Finally, you can use torchviz to visualize the tensor you are calculating the gradients of and ensure that the tensor you are accessing the .grad of is a leaf in the computation graph.