I have made available a slide deck with some details about PyTorch internals in case you’re interested.
There are also two other articles I wrote about it in case you’re interested:
- PyTorch 1.0 tracing JIT and LibTorch C++ API to integrate PyTorch into NodeJS
- PyTorch – Internal Architecture Tour
Hope it helps !