I was wondering where (or how) I should approach try to understand PyTorch’s source code? I’m particularly interested in understanding the implementation of the autograd module in PyTorch. I’ve read through all of the docs in regards to autograd and Variables, I also have a good understanding of how the back propagation equations are derived. Though they didn’t help me much in understanding how the modules are implemented. As a result I’ve just been using a debugger to try to step through a toy example, though even then I get lost in the complexity of it all. Am I going about this the wrong way? Or if anyone has attempted this would they care so share how they approached it?
These two posts will help get through the code structure and build system:
It doesn’t touch the autograd system though.
The autograd system is moved into C now, and is multi-threaded, so stepping through the python debugger is probably a bit pointless.
Here’s a pointer to very old source code, where all the logic was in Python, in the early days of PyTorch. The logic is largely the same still, but moved into C.
Thanks I’ll give it a read!
I also did an article about it here.