Hey guys,
As many people here (and there) once I trained my network I needed to run it in C++. Actually, my final aim is to run my network on the GPU, and running it from C++ on CPU is an intermediate validation step.
I’ve stumbled upon a few projects that presumably let you do the work in question but I found them overly complex to understand how to actually implement this convolution or batch norm layers than I need to implement on GPU.
So I implemented my own mini-framework I want to share with you that lets you dump some pyTorch modules, easily load them into C++ and evaluate it. My primary purpose of sharing the code with you is educational. If you ever have a need to implement neural net inference be it on CPU, GPU or any other specific kind of hardware you can always use my code as reference. I’ve checked that it produces extremely similar output to what pyTorch outputs. It does not support entire pyTorch but only selected, most basic modules that I needed. These are: ReLU, Sigmoid, Linear, Conv2d, BatchNorm2d.
Here is a link to my framework: https://github.com/maxest/MaxestFramework
And here is the code related to neural networks: https://github.com/maxest/MaxestFramework/tree/master/src/common
Only files neural_network.h and neural_network.cpp are relevant.
Here https://github.com/maxest/MaxestFramework/tree/master/tools/neural_network is a python script torch_layers_dumper.py that lets you dump a whole sequential module and save it to txt file. There is also a script file neural_network_test.py that creates a simple model and dumps it. The model is already there for convenience called model.txt. There is a helper function called UnitTest in neural_network.h/cpp that takes as input path to this model.txt and evaluates it. It ouputs the same values as the test python script neural_network_test.py.
I hope you will find this code useful. If you have any questions feel free to ask.
Cheers