# PyTorch Gradients

**URL:** <https://discuss.pytorch.org/t/pytorch-gradients/884>\
**Category:** Uncategorized\
**Created:** [March 5, 2017, 8:24am UTC](https://discuss.pytorch.org/t/pytorch-gradients/884 "2017-03-05T08:24:54Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\
**Post date:** [March 5, 2017, 9:27am UTC](https://discuss.pytorch.org/t/pytorch-gradients/884/2 "2017-03-05T09:27:34Z")

</div>

Hi,

When you perform `loss.backward()` the gradients are accumulated inplace in each Variable that requires gradient.  
That is why you need to perform `optimizer.zero_grad()` before each backward.

If you want to accumulate gradients from multiple backwards, you can just backward multiple times without resetting the gradients:

```auto
optimizer.zero_grad()

for i in range(minibatch):
    loss = model(batch_data[i])
    loss.backward()

optimizer.step()

```

---

_[View the full topic](https://discuss.pytorch.org/t/pytorch-gradients/884)._