# Not able to understand autograd

**URL:** <https://discuss.pytorch.org/t/not-able-to-understand-autograd/15391>\
**Category:** autograd\
**Created:** [March 24, 2018, 9:06am UTC](https://discuss.pytorch.org/t/not-able-to-understand-autograd/15391 "2018-03-24T09:06:27Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![prateekagarwal3](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/prateekagarwal3/32/3183_2.png) [@prateekagarwal3](https://discuss.pytorch.org/u/prateekagarwal3)\
**Post date:** [March 24, 2018, 9:06am UTC](https://discuss.pytorch.org/t/not-able-to-understand-autograd/15391/1 "2018-03-24T09:06:27Z")

</div>

I was reading the official tutorial on autograd (link : [http://pytorch.org/tutorials/beginner/blitz/autograd\_tutorial.html](http://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html)) in this once the out.backward() is performed and I tried printing values of z.grad and y.grad I got a None, shouldn’t z.grad have a not None value and the same for y.grad?

---

<div class="post-metadata">

**Author:** ![jpeg729](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jpeg729/32/3464_2.png) [@jpeg729](https://discuss.pytorch.org/u/jpeg729)\
**Post date:** [March 24, 2018, 9:48am UTC](https://discuss.pytorch.org/t/not-able-to-understand-autograd/15391/2 "2018-03-24T09:48:35Z")

</div>

For easy reference, here is the code.

```auto
x = Variable(torch.ones(2, 2), requires_grad=True)
y = x + 2
z = y * y * 3
out = z.mean()
out.backward()

```

PyTorch does not expose the gradients of `y` and `z` because they are not “leaf” Variables.

`x` is a “leaf” Variable because you created it using Variable(…).  
`y`, `z` and `out` are not a “leaf” Variables because the are the results of calculations.

---

<div class="post-metadata">

**Author:** ![prateekagarwal3](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/prateekagarwal3/32/3183_2.png) [@prateekagarwal3](https://discuss.pytorch.org/u/prateekagarwal3)\
**Post date:** [March 24, 2018, 10:12am UTC](https://discuss.pytorch.org/t/not-able-to-understand-autograd/15391/3 "2018-03-24T10:12:01Z")

</div>

By “does not expose” does it mean that PyTorch does not allow us to print it by print(x.grad)? But I hope it does get calculated right? and the gradients are back-propagated?

---

<div class="post-metadata">

**Author:** ![jpeg729](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jpeg729/32/3464_2.png) [@jpeg729](https://discuss.pytorch.org/u/jpeg729)\
**Post date:** [March 24, 2018, 10:27am UTC](https://discuss.pytorch.org/t/not-able-to-understand-autograd/15391/4 "2018-03-24T10:27:22Z")

</div>

`y.grad` gets calculated but once it has been used it is discarded to save memory. How else would backprop work?

Others have asked how to get the gradient with respect to `y`. [How to compute the gradients of non leaf variables in PyTorch](https://discuss.pytorch.org/t/how-to-compute-the-gradients-of-non-leaf-variables-in-pytorch/5736)

---

<div class="post-metadata">

**Author:** ![prateekagarwal3](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/prateekagarwal3/32/3183_2.png) [@prateekagarwal3](https://discuss.pytorch.org/u/prateekagarwal3)\
**Post date:** [March 24, 2018, 10:33am UTC](https://discuss.pytorch.org/t/not-able-to-understand-autograd/15391/5 "2018-03-24T10:33:52Z")

</div>

Okay got the point, so only the leaf variables grad is stored and available for display? and the grad for the non leaf variables is only calculated when needed and then discarded?

---

<div class="post-metadata">

**Author:** ![prateekagarwal3](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/prateekagarwal3/32/3183_2.png) [@prateekagarwal3](https://discuss.pytorch.org/u/prateekagarwal3)\
**Post date:** [March 24, 2018, 10:35am UTC](https://discuss.pytorch.org/t/not-able-to-understand-autograd/15391/6 "2018-03-24T10:35:16Z")

</div>

Thanks for the help @jpeg729