How to save torchscript model using cpp?

SixerWang · November 1, 2019, 6:17am

I know detail about how to save torchscript model using python, and load it with c++ from the doc: link.

But how to save torchscript using cpp？I should train my model using c++, but I don’t find any detail about it in document.
Thanks!

21fa417e3fb06e56040c · November 5, 2019, 1:19am

Hi.

You can refer to the repository below to do what you want.

First create a C ++ model with the same structure as the Python model
Save your Python model as a JIT script.
Instead of loading the model into a JIT script, you use torch :: load to load it.

Created by referring torchvision.

github.com

pytorch/vision/blob/cd1748443afb1ccd168a3b3da29317ca5d4cd253/test/test_models.cpp#L10


#include <torch/script.h>
#include <torch/torch.h>
#include <iostream>


#include "../torchvision/csrc/models/models.h"


using namespace vision::models;


template <typename Model>
torch::Tensor forward_model(const std::string& input_path, torch::Tensor x) {
  Model network;
  torch::load(network, input_path);
  network->eval();
  return network->forward(x);
}


torch::Tensor forward_alexnet(const std::string& input_path, torch::Tensor x) {
  return forward_model<AlexNet>(input_path, x);
}

SixerWang · November 5, 2019, 1:58am

Hi, Thanks for your reply.
But I should train model with C++ and save the model as JIT script. The reason why I should save model as JIT script is that the inference code can’t access to model structure(the model defination class). Thus, I should find a way to save model as JIT script using C++.
However, the code you provide above should pass Model instance as parameter, so It doesn’t meet my need.

21fa417e3fb06e56040c · November 5, 2019, 2:04am

I don’t know
Is it to avoid exposing the model structure in inference?

SixerWang · November 5, 2019, 2:12am

It is one aspect. On the other hand, the Script Model has high-performance.

abtion · November 18, 2019, 11:34am

Hi, have you solved it? I have the same problem, and I try to trian the script model in C++, but it not works, could you please give me some help while you have train it successfully?thanks

abtion · November 20, 2019, 8:12am

Hi, I found that both torchscript and torch::nn::Module model file can be loaded with torch::load(model, "model_file.pt");. It isn’t nesscery to save torchscript model in cpp while you load torchscript model to nn::Module and train it.

SixerWang · November 27, 2019, 2:02am

link : (libtorch) Save MNIST c++ example's trained model into a file, and load in from another c++ file to use for prediction? - #9 by Ziri_Ziri

SixerWang · December 10, 2019, 8:11am

@yf225 I am sorry to disturb, Could u give me a little hint about whether libtorch could save torchscript model using cpp or not. Thank you so much!

yf225 · December 10, 2019, 5:10pm

@SixerWang By “saving a TorchScript model in C++”, do you mean one of the following:

You want to embed Python/TorchScript code in your C++ code, and then save that portion of code to a TorchScript model file
Answer: This is not supported right now, but it is possible to implement / expose this API. I’d suggest opening an issue in https://github.com/pytorch/pytorch/issues and we can discuss the use cases.
You want to save a model written in C++ frontend to a TorchScript model file
Answer: This is not supported and likely won’t be supported in the short term, because implementing this mechanism is a huge effort, and we will need more use cases to actually establish this as a project to work on.

SixerWang · December 11, 2019, 5:15am

Thank you very much for your reply. My situation is the second one. I will change my training process.Thank you!

freezek · October 4, 2020, 1:56pm

Hi, have you solved the problem?