List of tensor as parameter of forward()

C++
1

Hello,

I’d like to know that how to input a list of tensors as a parameter of forward() function in C++ (libtorch).

I designed forward() function of my model as below and generated libtorch model using Torchscript.
Such processes were successfully done.

    def forward(self, X: List[torch.Tensor], points : List[torch.Tensor]):

However, I tried to use libtorch model as below in C++, but I failed because of invalid type matching.

Status
LibTorchBackend::Context::Execute(
    std::vector<std::vector<torch::jit::IValue>>* inputs_,
    std::vector<torch::Tensor>* outputs_)
{ ...
model_outputs_ = torch_model_->forward(*inputs_);
...}

/workspace/src/backends/pytorch/libtorch_backend.cc:862:52: error: no matching function for call to 'torch::jit::Module::forward(std::vector<std::vector<c10::IValue> >&)'
     model_outputs_ = torch_model_->forward(*inputs_);

As I know, the type of parameter of forward() function is std::vector<torch::jit::IValue> .
However, in my case, I have two list type of parameters: X and points.
Thus, I tried std::vector<std::vector<torch::jit::IValue>>.

Please give me an idea to solve this problem. :sob:
Thanks

Edit:

I also tried below approach in order to make a list of IValue,

std::vector<torch::jit::IValue> input;
... // put some data into the vector input 
torch::jit::IValue data_input = torch::jit::IValue(input);

This also generated errors

/opt/tritonserver/include/torch/ATen/core/List_inl.h: In instantiation of 'c10::List<T>::List() [with T = c10::IValue]':
/workspace/src/backends/pytorch/libtorch_backend.cc:690:26:   required from here
/opt/tritonserver/include/torch/ATen/core/List_inl.h:20:3: error: static assertion failed: This constructor is not valid for List<IValue>. Please use c10::impl::GenericList(elementType) instead.
   static_assert(!std::is_same<T, IValue>::value, "This constructor is not valid for List<IValue>. Please use c10::impl::GenericList(elementType) instead.");
2 
torch::List<torch::Tensor> x;
x.push_back(a);
x.push_back(b);
std::vector<torch::jit::IValue> jit_inputs(0);
jit_inputs.emplace_back(x);
model.forward(jit_inputs);