Torchscript C++ forward() Expected a value of type 'List[Tensor]' for argument 'images'

jit
1

Hello Everyone,

I have been trying to figure out an issue with torchscript in c++

When I use the scripted model in python it works

# code in python for inference
script = torch.jit.optimize_for_inference(torch.jit.script(model))
out = script([torch.ones(3, 224, 224).cuda()])

When I load the scripted model in C++

// code in c++ for inference
torch::jit::script::Module module = torch::jit::load("scripts/engine1.pt", torch::kCUDA);

std::vector<torch::jit::IValue> inputs;
inputs.push_back(torch::ones({ 1, 3, 224, 224 }).to(torch::kCUDA));

at::Tensor output = module.forward(inputs).toTensor();

I get this error

forward() Expected a value of type 'List[Tensor]' for argument 'images' but instead found type 'Tensor'.
Position: 1
Declaration: forward(__torch__.torchvision.models.detection.faster_rcnn.___torch_mangle_41.FasterRCNN self, Tensor[] images, Dict(str, Tensor)[]? targets=None) -> ((Dict(str, Tensor), Dict(str, Tensor)[]))
Exception raised from checkArg at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen/core/function_schema_inl.h:339 (most recent call first):
00007FF9DD50FA7A00007FF9DD50E630 c10.dll!c10::initLogging [<unknown file> @ <unknown line number>]
00007FF9DD50F42A00007FF9DD50E630 c10.dll!c10::initLogging [<unknown file> @ <unknown line number>]
00007FF9DD510B2100007FF9DD50E630 c10.dll!c10::initLogging [<unknown file> @ <unknown line number>]
00007FF9DD51074500007FF9DD50E630 c10.dll!c10::initLogging [<unknown file> @ <unknown line number>]
00007FF9DD50DACF00007FF9DD50DA60 c10.dll!c10::Error::Error [<unknown file> @ <unknown line number>]
00007FF9DD50C33600007FF9DD50C270 c10.dll!c10::detail::torchCheckFail [<unknown file> @ <unknown line number>]
00007FF999803CF800007FF9998017E0 torch_cpu.dll!caffe2::TypeMeta::_typeMetaData<caffe2::detail::_Uninitialized> [<unknown file> @ <unknown line number>]
00007FF99980363B00007FF9998017E0 torch_cpu.dll!caffe2::TypeMeta::_typeMetaData<caffe2::detail::_Uninitialized> [<unknown file> @ <unknown line number>]
00007FF9998B612700007FF9998B60D0 torch_cpu.dll!torch::jit::Function::operator() [<unknown file> @ <unknown line number>]
00007FF99FCDDC4A00007FF99FCDDA00 torch_cpu.dll!torch::jit::Method::operator() [<unknown file> @ <unknown line number>]
00007FF99FA848E400007FF99FA847E0 torch_cpu.dll!torch::jit::Module::forward [<unknown file> @ <unknown line number>]
00007FF6CBE113E500007FF6CBE11090 server.exe!main [C:\Users\AP9Xa\Desktop\projects\server\proj\server\server.cpp @ 14]
00007FF6CBE0D78900007FF6CBE0D750 server.exe!invoke_main [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 79]
00007FF6CBE0D62E00007FF6CBE0D500 server.exe!__scrt_common_main_seh [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288]
00007FF6CBE0D4EE00007FF6CBE0D4E0 server.exe!__scrt_common_main [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 331]
00007FF6CBE0D81E00007FF6CBE0D810 server.exe!mainCRTStartup [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp @ 17]
00007FFA4425734400007FFA44257330 KERNEL32.DLL!BaseThreadInitThunk [<unknown file> @ <unknown line number>]
00007FFA448A26B100007FFA448A2690 ntdll.dll!RtlUserThreadStart [<unknown file> @ <unknown line number>]

I would really appreciate the help

1 Like