Since my network (rnn used) does not converge, I want to see the gradient of the weights of each layer. I tried using tensor.grad to get the gradient, however, the output is always None. How can I print the gradient in each layer? Thanks
2 Likes
Once you’ve called backward
to calculate the gradients, you can directly print them using something like this:
model = nn.Sequential(
nn.Linear(10, 2)
)
...
loss.backward()
print(model[0].weight.grad)
In your case the model definition will look a bit different. So depending how you’ve implemented the model, you might need to index the layers like in my example or call the layers directly with their names:
print(model.my_layer.weight.grad)
22 Likes
@thank you so much. Problem solved
Can you tell me how you finally get the gradients of each layer?