Accessing parameter grads as a list

If I want a network’s parameters, I can easily throw them into a list using

params = list(network.parameters())

Is there an easy way to do this for the gradients of the network parameters? I know each gradient can be accessed using grad = param.grad, but storing all these gradients in an array requires me to iterate over each parameter in the network and access its .grad. Any help is appreciated here!

Hi,

You can use grads = autograd.grad(loss, network.parameters()) and it will return to you a list of all the gradients.
If you already did a .backward() that populated the .grad fields, you can do grads = [p.grad for p in network.parameters()] as well.

2 Likes