Where can i find the related process in the source code?
So this has several parts (and take my answer with a grain of salt, I am by no means an expert in PyTorch internals).
- The actual back-propagation is eventually done in torch/csrc/autograd/engine.cpp. There also is a python_engine.cpp in the same directory which sits between the torch/autograd python code and the engine.cpp.
- The backward objects are (and can only be) instantiated in the C interface. The definitions are autogenerated during the build, they end up in torch/csrc/autograd/generated and are generated by and from the files in tools/autograd. There you’ll find templates C code, yaml files listing the functions to be generated and their backwards, and the actual generation code.
Best regards
Thomas