Deploy c++ trained model on PyTorch

C++
1

I have trained a model using GPU c++ frontend to Pytorch. Now I want to deploy the model (for inference only) on a CPU-only linux dist (actually, on a RaspBerry Pi).

I have a trained torch::nn::Module (created using c++ front-end) that I saved using torch::save. It was trained on a cluster with GPU support. Now I want to take the trained model, and deploy it on a RaspBerry Pi.

I have read a lot of tutorials on deployment, but many seem to assume that training happened in Python. Now I have trouble seeing the forest for the trees. Where to start?

2

Well, the basic idea is:
-> load your network
-> load your trained weight into your loaded network
-> load your input data
-> pre-process it
-> feed the input data to your network
-> post-process it
-> show the results

here is an example that i can find
https://github.com/weixu000/libtorch-yolov3-deepsort/tree/master/detection/src.
this example is using torch::nn. Note that to load the trained weight into the network, he write his own load function. So, to load the trained weight into the network, you can use torch::load function.

And I have some suggestions for you:

  1. Since you are using RaspBerry Pi, why don’t you use python (instead of using C++) to build your apps?
  2. If you really need to build your apps in C++, maybe you can train your network in python and use TorchScript features from PyTorch. This features is very helpful on deployment.
    here are some examples on how to use torchscript:
    -> https://github.com/BIGBALLON/PyTorch-CPP/blob/master/prediction.cpp
    -> https://www.programmersought.com/article/5464871369/

if this answer still doesn’t help, you can start by trying some examples in the documentation (https://pytorch.org/docs/stable/cpp_index.html). these examples really help me in understanding how the pytorch c ++ api works
regards,
Albert Christianto