# Whats the difference between Optimizer.zero\_grad() vs nn.Module.zero\_grad()

**URL:** <https://discuss.pytorch.org/t/whats-the-difference-between-optimizer-zero-grad-vs-nn-module-zero-grad/59233>\
**Category:** Uncategorized\
**Created:** [October 25, 2019, 6:35pm UTC](https://discuss.pytorch.org/t/whats-the-difference-between-optimizer-zero-grad-vs-nn-module-zero-grad/59233 "2019-10-25T18:35:33Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![banikr](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/banikr/32/54353_2.png) [@banikr](https://discuss.pytorch.org/u/banikr)\
**Post date:** [October 25, 2019, 6:35pm UTC](https://discuss.pytorch.org/t/whats-the-difference-between-optimizer-zero-grad-vs-nn-module-zero-grad/59233/1 "2019-10-25T18:35:33Z")

</div>

which one is used when?

[https://pytorch.org/docs/stable/optim.html](https://pytorch.org/docs/stable/optim.html) here it uses Optimizer.zero\_grad()

[https://github.com/pytorch/examples/blob/master/dcgan/main.py](https://github.com/pytorch/examples/blob/master/dcgan/main.py) codes here use NetG.zero\_grad().

I know that optimizer.zero\_grad() makes the existing gradients zero before backpropagating the loss. Then update network parameters. What is nn.Module.zero\_grad() used for?

---

<div class="post-metadata">

**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\
**Post date:** [October 25, 2019, 7:02pm UTC](https://discuss.pytorch.org/t/whats-the-difference-between-optimizer-zero-grad-vs-nn-module-zero-grad/59233/2 "2019-10-25T19:02:42Z")

</div>

The nn.Module.zero\_grad() also sets the gradients to 0 for all parameters.

If you ceated your optimizer like `opt = optim.SGD(model.paremeters(), xxx)`, then `opt.zero_grad()` and `model.zero_grad()` will have the same effect.  
The distinction is useful for people that have multiple models in the same optimizer, or multiple optimizer for different part of their model.