# Multiple call to backward still works when retain\_graph=false, why?

**URL:** <https://discuss.pytorch.org/t/multiple-call-to-backward-still-works-when-retain-graph-false-why/9475>\
**Category:** autograd\
**Created:** [November 3, 2017, 7:08pm UTC](https://discuss.pytorch.org/t/multiple-call-to-backward-still-works-when-retain-graph-false-why/9475 "2017-11-03T19:08:35Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![wcsday](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/w/e79b87/32.png) [@wcsday](https://discuss.pytorch.org/u/wcsday)\
**Post date:** [November 3, 2017, 7:08pm UTC](https://discuss.pytorch.org/t/multiple-call-to-backward-still-works-when-retain-graph-false-why/9475/1 "2017-11-03T19:08:35Z")

</div>

The tutorial says( [http://pytorch.org/tutorials/beginner/former\_torchies/autograd\_tutorial.html#gradients](http://pytorch.org/tutorials/beginner/former_torchies/autograd_tutorial.html#gradients)): so if you even want to do the backward on some part of the graph twice, you need to pass in retain\_variables = True during the first pass

The example it gives is:

import torch  
from torch.autograd import Variable  
x = Variable(torch.ones(2, 2), requires\_grad=True)  
y = x + 2  
y.backward(torch.ones(2, 2), retain\_graph=True)  
print(x.grad)  
z = y \* y  
print(z)  
gradient = torch.randn(2, 2)  
y.backward(gradient)  
print(x.grad)

But when I try this code with retain\_graph=True and retain\_graph=False, they both works with no error, and the gradients are corrects.  
Anything wrong with the example?

Thanks!

---

<div class="post-metadata">

**Author:** ![colesbury](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/colesbury/32/26_2.png) [@colesbury](https://discuss.pytorch.org/u/colesbury)\
**Post date:** [November 4, 2017, 3:13am UTC](https://discuss.pytorch.org/t/multiple-call-to-backward-still-works-when-retain-graph-false-why/9475/2 "2017-11-04T03:13:37Z")

</div>

In this specific case you do not need `retain_graph=True`, but in general you may need it. As you compute the forward pass, PyTorch saves variables that will be needed to compute the gradients in the backward pass. For example, `z = y * y` needs to save the value of `y`, because `dz/dy = 2*y` (or `y + y`). However, `y = x + 2` doesn’t need to save anything because `dy/dx = 1` which doesn’t depend on x.

When you call `backwards()` with `retain_graph=False` (or without specifying it), the automatic differentiation engine frees the saved variables as it computes the gradients. If you call `backwards()` again, it will fail with an exception if it needs any freed saved variables. If it doesn’t need any saved variables, like in your example, then it will succeed, but you shouldn’t rely on this behavior.

If you change `y = x + 2` to `y = x * x`, you will see an error:

```auto
Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

```

---

<div class="post-metadata">

**Author:** ![wcsday](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/w/e79b87/32.png) [@wcsday](https://discuss.pytorch.org/u/wcsday)\
**Post date:** [November 7, 2017, 1:51am UTC](https://discuss.pytorch.org/t/multiple-call-to-backward-still-works-when-retain-graph-false-why/9475/3 "2017-11-07T01:51:57Z")

</div>

Thanks for the explanation!