I implemented a method for computing optical flow (Spynet).
a) How does your software stack look like?
I implemented a Custom dataset loader which uses OpenCV methods (imread, readOpticalFlow) to read images and corresponding optical flow from the FlyingChairs dataset. Training data is then augmented (with more OpenCV methods) and finally converted to torch Tensors.
The tensors are fed into the CNN.
b) And how does the PyTorch C++ API interact with other components in your stack?
I don’t understand the question.
Are you using JIT (TorchScript) models or torch::nn models?
No.
Are you training models in C++?
Yes.
How do you think the PyTorch C++ API (or the ecosystem in general) should be improved?
When I started to learn PyTorch C++ I had no experience with PyTorch or CNNs. I found it difficult to find out what each function does since the documentation seems to be auto-generated from Code without doxygen-style comments. I had to constantly look up things in the Python interface docs. So I think the docs could be improved in a first step by linking the corresponding docs of the Python interface.
Also I’ve seen Python code which selects for example [,,*,0] from a tensor of shape (N,W,H,C). I’d like to do that in C++ but I don’t know how (or if it’s possible).