How to load previously saved model in cpp frontend?

Hello, I was trying to load a previous model which was saved using a python script in cpp frontend but got some error. Also is there a way to use load_state_dict function which is present in python to load the saved state dict in cpp?
thanks in advance.

Here is actually a nice explanation by @yf225

Should answer your question as well. Please report back if not.

@Ronit-j Besides trying out the post that Martin linked, could you also share the error you see, and the code to reproduce the error?

We are working on adding load_state_dict to C++ frontend, but one caveat is that the exact same model structure needs to also be defined in C++ frontend (i.e. you’ll need to write C++ code that defines the same model structure as the Python model). If this is undesirable, you can also trace the model using JIT and then load it into C++ frontend (https://pytorch.org/tutorials/advanced/cpp_export.html is a good guide on this).

1 Like

I tried running the example that you referred I get this error https://paste.ubuntu.com/p/3KMv7xBwKc/

Thank you I’ll try it and let you know.

I think due to some changes torch::jit::load does not return a shared pointer anymore. Check for example the tests. You would need to replace every shared pointer like so

// Replace
std::shared_ptr<torch::jit::script::Module> module = torch::jit::load(module_path);
// by this
torch::jit::script::Module module = torch::jit::load(module_path);

// replace every -> by .
// for example
torch::Tensor prediction = module->forward(inputs).toTensor();
// by 
torch::Tensor prediction = module.forward(inputs).toTensor();

I am not too familiar with the inner workings here so I dont know why this was done. But it will fix your problem. The tutorial linked earlier, is also currently being updated according to this pull request.

1 Like