How can I get the gradients of the weights of each layer?

Since my network (rnn used) does not converge, I want to see the gradient of the weights of each layer. I tried using tensor.grad to get the gradient, however, the output is always None. How can I print the gradient in each layer? Thanks

2 Likes

Once you’ve called backward to calculate the gradients, you can directly print them using something like this:

model = nn.Sequential(
    nn.Linear(10, 2)
)
...
loss.backward()
print(model[0].weight.grad)

In your case the model definition will look a bit different. So depending how you’ve implemented the model, you might need to index the layers like in my example or call the layers directly with their names:

print(model.my_layer.weight.grad)
21 Likes

@thank you so much. Problem solved

Can you tell me how you finally get the gradients of each layer?