It is hard for me to understand all the components, structure, and the history of PyTorch project.
I know there is Torch library that works with Lua. Somewhere there, there’s C part of the code, that works with CUDA (but CUDA is written in C++, so how it is?). Suddenly some people took some part of these things and created PyTorch. There is also something like ATen that seems to be important and is somehow rewritten part of Torch (torch7?) to C++
I can’t get all these things. Is there any place or description of all the dependencies, components, and history of the development? I am especially interested in Pytorch, Torch, torch7, ATen, <TH/TH.h> <torch/torch.h>, CUDA, CuDNN.
The initial torch7 (which is what you call Torch) library was written in lua and based on a pure C library called TH for cpu computation and cpp library called THC for gpu computations (this include all the cuda/cudnn dependancies). There was also THNN and THCNN for neural network specific ops. These C/cpp libraries are the one providing TH/TH.h or THC/THC.h (that should only be used internally by the way).
When pytorch was created, it was initially only a python wrapper + new autograd engine + TH, THC, THNN, THCUNN.
Given the complexity of all the interfacing with TH* libraries, Aten was created to provide a nice cpp interface to all of these. Note that now, new functions are implemented directly into Aten and not the TH* libraries.
More recently, to be able to do cpp-only tasks, libtorch has been introduced as a replacement for the python wrapper. It is a pure cpp library that replaces it. And it’s main include file is torch/torch.h.
Hope this helps, let me know if I missed anything!