# In optimizer.zero\_grad(), set p.grad = None?

**URL:** <https://discuss.pytorch.org/t/in-optimizer-zero-grad-set-p-grad-none/31934>\
**Category:** Uncategorized\
**Created:** [December 12, 2018, 9:44am UTC](https://discuss.pytorch.org/t/in-optimizer-zero-grad-set-p-grad-none/31934 "2018-12-12T09:44:20Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\
**Post date:** [December 12, 2018, 10:25am UTC](https://discuss.pytorch.org/t/in-optimizer-zero-grad-set-p-grad-none/31934/2 "2018-12-12T10:25:11Z")

</div>

Hi,

There is one main difference:  
The main difference is that the Tensor containing the gradients will not be reallocated at every backward pass. Since memory allocation is quite expensive (especially on GPU), this is much more efficient.

There are other subtle differences between the two like some optimizers that behave differently if a gradient is 0 or None. It am sure there are other places that behave like that.

---

_[View the full topic](https://discuss.pytorch.org/t/in-optimizer-zero-grad-set-p-grad-none/31934)._