Get the gradient of the network parameters

Guys, I am stucking on getting the gradients of a network’s parameters. So basically, what I wanna do is creating a long one-dimensional tensor which stores all the gradients of the parameters(weights, bias) of a network without updating the parameters with Optimizer_Network.step(), the reason why I have to use this one-dimensional tensor is that I have to do some other operations later with it. How can I get it?

2 Likes

You could iterate all parameters and store each gradient in a list:

model = models.resnet50()
# Calculate dummy gradients
model(torch.randn(1, 3, 224, 224)).mean().backward()
grads = []
for param in model.parameters():
    grads.append(param.grad.view(-1))
grads = torch.cat(grads)
print(grads.shape)
> torch.Size([25557032])
8 Likes

@ptrblck Thanks bro, It works!

Hi the solution of @ptrblck works for me as well, but is there a more efficient way to do this? Possibly without a for loop especially for networks with large number of parameters.

2 Likes