Convert PyTorch Models to C++

Hi Everyone,
Suppose I have a pre-trained classic CNN model (e.g., ResNet which is trained on a dataset) in PyTorch. Now, I want to convert this model to C++ (fro some reasons like better performance). Could you help me please to do that?
Is the PyTorch ATen (A TENsor library) solution of this work?

1 Like

There is no directly C++ support for pytorch’s model. May be can export to an onnx model , deploy this mode using caffe2.

Dear @Teaonly,
Thank you for your answer. But in my opinion the syntax of caffe2 framework is so hard and its tutorials & docs are really bad. Is another solution exists for the above-mentioned task?
What is your opinion about Pytorch C++ Library?

1 Like

Have you try https://github.com/warmspringwinds/pytorch-cpp.
When I compiled pytorch-cpp, I got errors:

pytorch-cpp/src/pytorch.cpp:341:58: error: ‘Threshold_updateOutput’ was not declared in this scope

Any advise? Thanks.

@Hyer_Chen,
No, I haven’t use the above-mentioned repository. You can report your error in the Issue section of that repository. Maybe the below repository also be interesting for you:


However, I didn’t use that again!
Note that in the near future, maybe the PyTorch team release a PyTorch like C++ Interface. You can visit the below links:
1- https://github.com/pytorch/pytorch/issues/3335
2- https://github.com/caffe2/caffe2/issues/2439
3- https://github.com/pytorch/pytorch/issues/6032

Thanks. I think there may be some head file missing. And I am really looking forward to the “PyTorch team release a PyTorch like C++ Interface”.

1 Like

Caffe 2 is now merged into PyTorch, so the C++ inference should be available in future, theoretically

I think this link would be helpful:
Deep-Learning-in-Production

You can always try using my implementation of inference Eval model in C++ code :slight_smile:
It implements only a few modules (ReLU, Sigmoid, Linear, Conv2D, BatchNorm2D) but that should be easy to extend or you can just simply use it to implement your own inference from scratch.