How can I feed the output of one module to another in C++?

Hello everyone. I’m trying a very basic example, in which I’m loading an MTCNN model (consists of 3 submodules onet, pnet and rnet) and trying to feed forward them. This is the code :

#include <torch/torch.h>
#include <torch/script.h> // One-stop header.
#include <iostream>
#include <memory>
#include <cstdlib>

int main(int argc, const char* argv[]) 
{

    // Create a vector of inputs.
    std::vector<torch::jit::IValue> inputs;
    inputs.push_back(torch::ones({ 1, 3, 112, 112 }));

    torch::jit::script::Module onet;
    torch::jit::script::Module pnet;
    torch::jit::script::Module rnet;
    try 
    {
        std::string mtcnn_onet_path = "G:\\Codes\\python\\MTCNN\\weights_mtcnn_onet.pt";
        std::string mtcnn_pnet_path = "G:\\Codes\\python\\MTCNN\\weights_mtcnn_pnet.pt";
        std::string mtcnn_rnet_path = "G:\\Codes\\python\\MTCNN\\weights_mtcnn_rnet.pt";

        // Deserialize the ScriptModule from a file using torch::jit::load().
        onet = torch::jit::load(mtcnn_onet_path);
        pnet = torch::jit::load(mtcnn_pnet_path);
        rnet = torch::jit::load(mtcnn_rnet_path);

        // Execute the model and turn its output into a tensor.
        at::Tensor output_o = onet.forward(inputs).toTensor();
        at::Tensor output_p = pnet.forward(output_o).toTensor();
        at::Tensor output = rnet.forward(output_p).toTensor();

        std::cout << output.sizes() << std::endl;
        std::cout << "output: " << output[0] << std::endl;
    
    }
    catch (const c10::Error& e) 
    {
        std::cerr << "error loading the model\n";
        return -1;
    }

    std::cout << "ok\n";
    std::system("pause");
    return 0;
}

but I get :

Severity	Code	Description	Project	File	Line	Suppression State
Error	C2664	'c10::IValue torch::jit::Module::forward(std::vector<c10::IValue,std::allocator<c10::IValue>>)': cannot convert argument 1 from 'at::Tensor' to 'std::vector<c10::IValue,std::allocator<c10::IValue>>'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	54	

Severity	Code	Description	Project	File	Line	Suppression State
Error	C2664	'c10::IValue torch::jit::Module::forward(std::vector<c10::IValue,std::allocator<c10::IValue>>)': cannot convert argument 1 from 'at::Tensor' to 'std::vector<c10::IValue,std::allocator<c10::IValue>>'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	55	

Shouldnt we pass inputs as Tensors like in Python?
I couldnt find any documentation that properly explains this . any help in this regard is appreciated.

By the way these are the modules : https://gofile.io/d/DyeriF

Hi, the output_o had been converted back to an at::Tensor, and this is now placed inside pnet.forward().
And error is thrown at this line.
output_o needs to be converted to a torch::jit::IValue an then put inside pnet.forward().
The same goes to rnet.forward() as well.

1 Like

Thanks, really appreciate it, but what Should I be setting as the type for the new outputs?
using auto also fails! :

auto output_o = onet.forward(inputs);
auto output_p = pnet.forward(output_o);

while I expected this to work!
also these failed as well :

std::vector<c10::IValue, std::allocator<c10::IValue>> output_o = onet.forward(inputs);
std::vector<c10::IValue, std::allocator<c10::IValue>> output_p = pnet.forward(output_o);

and also this :

std::vector<c10::IValue> output_o = onet.forward(inputs);
std::vector<c10::IValue>  output_p = pnet.forward(output_o);

results in :

Severity	Code	Description	Project	File	Line	Suppression State
Error	C2440	'initializing': cannot convert from 'c10::IValue' to 'std::vector<c10::IValue,std::allocator<c10::IValue>>'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	52	
Error	C2440	'initializing': cannot convert from 'c10::IValue' to 'std::vector<c10::IValue,std::allocator<c10::IValue>>'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	53	

and also this:

torch::jit::IValue output_o = onet.forward(inputs);
torch::jit::IValue output_p = pnet.forward(output_o);

results in :

Severity	Code	Description	Project	File	Line	Suppression State
Error	C2664	'c10::IValue torch::jit::Module::forward(std::vector<c10::IValue,std::allocator<c10::IValue>>)': cannot convert argument 1 from 'c10::IValue' to 'std::vector<c10::IValue,std::allocator<c10::IValue>>'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	53	
Error	C2664	'c10::IValue torch::jit::Module::forward(std::vector<c10::IValue,std::allocator<c10::IValue>>)': cannot convert argument 1 from 'c10::IValue' to 'std::vector<c10::IValue,std::allocator<c10::IValue>>'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	54	

and

std::vector<torch::jit::IValue> output_o = onet.forward(inputs);
std::vector<torch::jit::IValue> output_p = pnet.forward(output_o);

results in :

Severity	Code	Description	Project	File	Line	Suppression State
Error	C2440	'initializing': cannot convert from 'c10::IValue' to 'std::vector<c10::IValue,std::allocator<c10::IValue>>'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	52	

Severity	Code	Description	Project	File	Line	Suppression State
Error	C2440	'initializing': cannot convert from 'c10::IValue' to 'std::vector<c10::IValue,std::allocator<c10::IValue>>'	LibtorchPort	G:\Codes\cpp\port\LibtorchPort\LibtorchPort\LibtorchPort.cpp	53	

Looking at the errors,
the output from the model.forward() will be a
torch::jit::IValue.
However the input to a model.forward() will be a std::vector<torch::jit::IValue>

Can you try this ?

auto output_o = onet.forward(inputs);

std::vector<torch::jit::IValue>temp_op;

temp_op.push_back(output_o);

auto output_p = pnet.forward(temp_op);
1 Like

Thanks I no more get that error instead I get an excpetion :

Unhandled exception at 0x00007FFF8523A308 in LibtorchPort.exe: Microsoft C++ exception: std::runtime_error at memory location 0x0000002FB98F7830.

pointing at the very beginning meaning onet.forward:

auto output_o = onet.forward(inputs);
auto  output_p = pnet.forward(std::vector<torch::jit::IValue>{ output_o });
at::Tensor output = rnet.forward(std::vector<torch::jit::IValue>{ output_p }).toTensor();

I also uploaded the mtcnn modules in case they are needed : https://gofile.io/d/DyeriF

2 Likes

Thanks I’ll get back to you if any progress from my side :slight_smile:

1 Like

thanks a lot. really appreciate it:)