# Get the gradient of the network parameters

**URL:** <https://discuss.pytorch.org/t/get-the-gradient-of-the-network-parameters/50575>\
**Category:** autograd\
**Created:** [July 14, 2019, 12:03pm UTC](https://discuss.pytorch.org/t/get-the-gradient-of-the-network-parameters/50575 "2019-07-14T12:03:10Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wesker\_Rongkai\_Ma](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wesker_rongkai_ma/32/14073_2.png) [@Wesker\_Rongkai\_Ma](https://discuss.pytorch.org/u/Wesker_Rongkai_Ma)\
**Post date:** [July 14, 2019, 12:03pm UTC](https://discuss.pytorch.org/t/get-the-gradient-of-the-network-parameters/50575/1 "2019-07-14T12:03:10Z")

</div>

Guys, I am stucking on getting the gradients of a network’s parameters. So basically, what I wanna do is creating a long one-dimensional tensor which stores all the gradients of the parameters(weights, bias) of a network without updating the parameters with Optimizer\_Network.step(), the reason why I have to use this one-dimensional tensor is that I have to do some other operations later with it. How can I get it?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [July 14, 2019, 12:28pm UTC](https://discuss.pytorch.org/t/get-the-gradient-of-the-network-parameters/50575/2 "2019-07-14T12:28:40Z")

</div>

You could iterate all parameters and store each gradient in a list:

```python
model = models.resnet50()
# Calculate dummy gradients
model(torch.randn(1, 3, 224, 224)).mean().backward()
grads = []
for param in model.parameters():
    grads.append(param.grad.view(-1))
grads = torch.cat(grads)
print(grads.shape)
> torch.Size([25557032])

```

---

<div class="post-metadata">

**Author:** ![Wesker\_Rongkai\_Ma](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wesker_rongkai_ma/32/14073_2.png) [@Wesker\_Rongkai\_Ma](https://discuss.pytorch.org/u/Wesker_Rongkai_Ma)\
**Post date:** [July 15, 2019, 12:33am UTC](https://discuss.pytorch.org/t/get-the-gradient-of-the-network-parameters/50575/3 "2019-07-15T00:33:08Z")

</div>

@ptrblck Thanks bro, It works!

---

<div class="post-metadata">

**Author:** ![mahf](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mahf/32/23741_2.png) [@mahf](https://discuss.pytorch.org/u/mahf)\
**Post date:** [May 4, 2020, 6:08pm UTC](https://discuss.pytorch.org/t/get-the-gradient-of-the-network-parameters/50575/4 "2020-05-04T18:08:37Z")

</div>

Hi the solution of @ptrblck works for me as well, but is there a more efficient way to do this? Possibly without a for loop especially for networks with large number of parameters.

---

<div class="post-metadata">

**Author:** ![Megh\_Bhalerao](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/megh_bhalerao/32/13547_2.png) [@Megh\_Bhalerao](https://discuss.pytorch.org/u/Megh_Bhalerao)\
**Post date:** [August 30, 2022, 4:06pm UTC](https://discuss.pytorch.org/t/get-the-gradient-of-the-network-parameters/50575/5 "2022-08-30T16:06:05Z")

</div>

Yes, I have the same question. Is there any way to get the gradients of the parameters directly from the optimizer object without accessing the original parameters through the model object?  
Thanks!

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [August 30, 2022, 5:06pm UTC](https://discuss.pytorch.org/t/get-the-gradient-of-the-network-parameters/50575/6 "2022-08-30T17:06:57Z")

</div>

Yes, you could use the stored references to the parameters and use them to check their gradients:

```python
for p in optimizer.param_groups[0]['params']:
    print(p.grad)

```

but I don’t know what kind of advantage it would give you.

---

<div class="post-metadata">

**Author:** ![Megh\_Bhalerao](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/megh_bhalerao/32/13547_2.png) [@Megh\_Bhalerao](https://discuss.pytorch.org/u/Megh_Bhalerao)\
**Post date:** [August 30, 2022, 8:12pm UTC](https://discuss.pytorch.org/t/get-the-gradient-of-the-network-parameters/50575/7 "2022-08-30T20:12:05Z")

</div>

Thanks a lot. I think the advantage that I was looking for is that I can directly access the parameters which are to be optimized if they are a subset of the entire set of model parameters.

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [August 30, 2022, 9:09pm UTC](https://discuss.pytorch.org/t/get-the-gradient-of-the-network-parameters/50575/8 "2022-08-30T21:09:23Z")

</div>

Ah OK, that makes sense. I was unsure about your use case as I thought you might be expecting a faster access or so.