After successfully converting pytorch model into C++, this line module_->forward(inputs) runs into error. Am I missing something to write inputs properly?
inputs contains four tensors premises, premises_length, hypotheses, hypotheses_length.
premises can be any vector with integers, for example, sentence “what a great day today” can be converted into vector {800, 1, 30, 200, 28} according to vocabulary built by myself. This vector has length of 5, thus premises_length has the number 5. This contains only one sample. Below code contains batch_size = 2 samples.
Same applies to hypotheses, hypotheses_length.
#include <iostream>
#include <torch/script.h>
#include <vector>
#include <algorithm>
using namespace std;
int main() {
//load model
string model_path = "/data/users/me/deploy-trace.pt";
std::shared_ptr<torch::jit::script::Module> module_ = torch::jit::load(model_path);
assert(module_ != nullptr);
std::cout << "load model ok\n";
int n = 2; //two input samples containing word_indices from vocabulary
std::vector<float> premise_vec = {{2, 6226, 3, 2, 6226, 3}}; //flatten 1D vector for creating torch::Tensor, which contains two samples' word_indices vector {2, 6226, 3}, each with length 3
std::vector<int> premise_shape = {3, 3}; //each 3 means the length of word_indices vector
std::vector<float> hypotheses_vec = {{2, 6226, 9337, 3, 2, 6226, 9337, 3 }}; //flatten 1D vector for creating torch::Tensor, which contains two samples' word_indices vector {2, 6226, 9337, 3}, each with length 4
std::vector<int> hypotheses_shape = {4, 4}; //each 4 means the length of word_indices
torch::Tensor premises = torch::from_blob(premise_vec.data(), {n, premise_shape[0]}); //must use flatten vectors' data()
torch::Tensor premises_length = torch::from_blob(premise_shape.data(), {n});
int max_len = *max_element(hypotheses_shape.begin(), hypotheses_shape.end());
torch::Tensor hypotheses = torch::from_blob(hypotheses_vec.data(), {n, max_len});
torch::Tensor hypotheses_length = torch::from_blob(hypotheses_shape.data(), {n}); //1D tensor
std::vector<torch::jit::IValue> inputs;
inputs.push_back(premises);
inputs.push_back(premises_length);
inputs.push_back(hypotheses);
inputs.push_back(hypotheses_length);
auto outputs = module_->forward(inputs).toTuple();
torch::Tensor out_tensor = outputs->elements()[1].toTensor();//NOTE: take tuple[1], tensor shape (n, 2)
return 0;
}
My Python code is here for the forward() function
def forward(self,
premises,
premises_lengths,
hypotheses,
hypotheses_lengths):
"""
Args:
premises: A batch of varaible length sequences of word indices
representing premises. The batch is assumed to be of size
(batch, premises_length).
premises_lengths: A 1D tensor containing the lengths of the
premises in 'premises'.
hypothesis: A batch of varaible length sequences of word indices
representing hypotheses. The batch is assumed to be of size
(batch, hypotheses_length).
hypotheses_lengths: A 1D tensor containing the lengths of the
hypotheses in 'hypotheses'.
Returns:
logits: A tensor of size (batch, num_classes) containing the
logits for each output class of the model.
probabilities: A tensor of size (batch, num_classes) containing
the probabilities of each output class in the model.
"""
Error messages are listed below
load model ok
terminate called after throwing an instance of 'std::runtime_error'
what():
The shape of the mask [2, 3] at index 1 does not match the shape of the indexed tensor [2, 0] at index 1 (invalid_mask at ../aten/src/ATen/native/Indexing.cpp:75)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x45 (0x7f7fc0d30575 in /data/users/me/nlpc/torch/lib/libc10.so)
frame #1: <unknown function> + 0x67cc61 (0x7f7fc15bfc61 in /data/users/me/nlpc/torch/lib/libcaffe2.so)
frame #2: <unknown function> + 0x67e150 (0x7f7fc15c1150 in /data/users/me/nlpc/torch/lib/libcaffe2.so)
frame #3: <unknown function> + 0x67fb00 (0x7f7fc15c2b00 in /data/users/me/nlpc/torch/lib/libcaffe2.so)
frame #4: at::native::index_put_(at::Tensor&, c10::ArrayRef<at::Tensor>, at::Tensor const&, bool) + 0x8b (0x7f7fc15c432b in /data/users/me/nlpc/torch/lib/libcaffe2.so)
frame #5: at::TypeDefault::index_put_(at::Tensor&, c10::ArrayRef<at::Tensor>, at::Tensor const&, bool) const + 0x70 (0x7f7fc1970770 in /data/users/me/nlpc/torch/lib/libcaffe2.so)
frame #6: torch::autograd::VariableType::index_put_(at::Tensor&, c10::ArrayRef<at::Tensor>, at::Tensor const&, bool) const + 0x963 (0x7f7fc4696353 in /data/users/me/nlpc/torch/lib/libtorch.so.1)
frame #7: <unknown function> + 0x9ba861 (0x7f7fc4a78861 in /data/users/me/nlpc/torch/lib/libtorch.so.1)
frame #8: <unknown function> + 0xa8d888 (0x7f7fc4b4b888 in /data/users/me/nlpc/torch/lib/libtorch.so.1)
frame #9: torch::jit::InterpreterState::run(std::vector<c10::IValue, std::allocator<c10::IValue> >&) + 0x1d (0x7f7fc4b4737d in /data/users/me/nlpc/torch/lib/libtorch.so.1)
frame #10: <unknown function> + 0xa735a3 (0x7f7fc4b315a3 in /data/users/me/nlpc/torch/lib/libtorch.so.1)
frame #11: ./test_libtorch() [0x4419a1]
frame #12: ./test_libtorch() [0x43a0eb]
frame #13: __libc_start_main + 0xf5 (0x7f7fc0222b35 in /usr/lib64/libc.so.6)
frame #14: ./test_libtorch() [0x43a859]
:
operation failed in interpreter:
Aborted