# How to understand PyTorch's source code

**URL:** <https://discuss.pytorch.org/t/how-to-understand-pytorchs-source-code/7600>\
**Category:** Uncategorized\
**Created:** [September 19, 2017, 9:16pm UTC](https://discuss.pytorch.org/t/how-to-understand-pytorchs-source-code/7600 "2017-09-19T21:16:32Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![cakeeatingpolarbear](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/c0e974/32.png) [@cakeeatingpolarbear](https://discuss.pytorch.org/u/cakeeatingpolarbear)\
**Post date:** [September 19, 2017, 9:16pm UTC](https://discuss.pytorch.org/t/how-to-understand-pytorchs-source-code/7600/1 "2017-09-19T21:16:33Z")

</div>

Hi all,  
I was wondering where (or how) I should approach try to understand PyTorch’s source code? I’m particularly interested in understanding the implementation of the autograd module in PyTorch. I’ve read through all of the docs in regards to autograd and Variables, I also have a good understanding of how the back propagation equations are derived. Though they didn’t help me much in understanding how the modules are implemented. As a result I’ve just been using a debugger to try to step through a toy example, though even then I get lost in the complexity of it all. Am I going about this the wrong way? Or if anyone has attempted this would they care so share how they approached it?

---

<div class="post-metadata">

**Author:** ![smth](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/smth/32/13_2.png) [@smth](https://discuss.pytorch.org/u/smth)\
**Post date:** [September 20, 2017, 4:27am UTC](https://discuss.pytorch.org/t/how-to-understand-pytorchs-source-code/7600/2 "2017-09-20T04:27:51Z")

</div>

These two posts will help get through the code structure and build system:

[http://pytorch.org/2017/05/11/Internals.html](http://pytorch.org/2017/05/11/Internals.html)

[http://pytorch.org/2017/06/27/Internals2.html](http://pytorch.org/2017/06/27/Internals2.html)

It doesn’t touch the autograd system though.  
The autograd system is moved into C now, and is multi-threaded, so stepping through the python debugger is probably a bit pointless.

Here’s a pointer to very old source code, where all the logic was in Python, in the early days of PyTorch. The logic is largely the same still, but moved into C.

> <https://github.com/pytorch/pytorch/blob/v0.1.1/torch/autograd/engine.py>

  

> <https://github.com/pytorch/pytorch/blob/v0.1.1/torch/autograd/function.py>

---

<div class="post-metadata">

**Author:** ![cakeeatingpolarbear](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/c/c0e974/32.png) [@cakeeatingpolarbear](https://discuss.pytorch.org/u/cakeeatingpolarbear)\
**Post date:** [September 20, 2017, 9:45am UTC](https://discuss.pytorch.org/t/how-to-understand-pytorchs-source-code/7600/3 "2017-09-20T09:45:19Z")

</div>

Thanks I’ll give it a read!

---

<div class="post-metadata">

**Author:** ![christianperone](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/christianperone/32/1054_2.png) [@christianperone](https://discuss.pytorch.org/u/christianperone)\
**Post date:** [May 4, 2018, 1:49pm UTC](https://discuss.pytorch.org/t/how-to-understand-pytorchs-source-code/7600/4 "2018-05-04T13:49:35Z")

</div>

I also did an article about it [here](http://blog.christianperone.com/2018/03/pytorch-internal-architecture-tour/).