Converting vector<double> to TorchScript model input

Question

How do I convert a std::vector<double> into a std::vector <torch::jit::IValue> which can be input to a TorchScript model?

Background

I trained a model in Python and followed this guide to use the model in C++: https://pytorch.org/tutorials/advanced/cpp_export.html#step-4-executing-the-script-module-in-c

Please note that I have almost no experience with C++

Working code with dummy Tensor

Everything works fine when I make a dummy Tensor directly in C++:

#include <torch/script.h> // One-stop header.

#include <iostream>
#include <memory>

using namespace std::chrono;

int main(int argc, const char* argv[]) {
    std::cout << "ok\n";
    if (argc != 2) {
        std::cerr << "usage: example-app <path-to-exported-script-module>\n";
        return -1;
    }


    torch::jit::script::Module module;
    try {
        // Deserialize the ScriptModule from a file using torch::jit::load().
        module = torch::jit::load(argv[1]);
    }
    catch (const c10::Error& e) {
        std::cerr << "error loading the model\n";
        return -1;
    }

    std::cout << "ok\n";

    // Create a vector of inputs.
    int sample_rate = 16000;
    int input = sample_rate * 100;
    std::vector<torch::jit::IValue> inputs;
    inputs.push_back(torch::ones({1, input}));

    // Execute the model and turn its output into a tensor.
    at::Tensor output = module.forward(inputs).toTensor().sigmoid();
    std::cout << output.slice(/*dim=*/1, /*start=*/0, /*end=*/5) << '\n';

Attempting the same with vector double

In our real application the audio data is of the format std::vector<double>, however. This cannot be changed.
From my Googling, what I would likely need is the function torch::from_blob(), so I tried to replace:

// Create a vector of inputs.
int sample_rate = 16000;
int input = sample_rate * 100;
std::vector<torch::jit::IValue> inputs;
inputs.push_back(torch::ones({1, input}));

with:

std::vector <torch::jit::IValue> inputs;
// dummy double
std::vector<double> wavData = std::vector<double>(5);
auto options = torch::TensorOptions().dtype(torch::kFloat32);
inputs.push_back(torch::from_blob(wavData, {1, input}, options));

Problem

The code I found only used these 3 arguments. But this throws the error:

path/example-app.cpp: In function ‘int main(int, const char**)’:
path/example-app.cpp:47:71: error: no matching function for call to ‘from_blob(std::vector<double, std::allocator<double> >&, <brace-enclosed initializer list>, c10::TensorOptions&)’ inputs.push_back(torch::from_blob(wavData, {1, input}, options));

It seems I also need to supply a const Deleter& deleter, but I have no clue what how I’m supposed to create that.
Unfortunately the docs page of from_blob() is broken and therefore also not of much help.

Any insights are appreciated!

@NumesSanguis
At first glance, should you use torch::kFloat64? Your raw data is doulbe. I did not test on my machine though.

@glaringlee You might have identified the issue coming after this! It does not solve the original issue, however.

This standalone script:

#include <torch/script.h> // One-stop header.
#include <vector>
#include <iostream>
#include <memory>

int main(int argc, char** argv)
{
    std::cout << "Start successful\n";
    std::vector<double> wavData = std::vector<double>(5);
    int len = wavData.size();
    auto options = torch::TensorOptions().dtype(torch::kFloat64);
    torch::from_blob(wavData, {1, len}, options);
}

Produces the following error:

to_tensor_test.cpp.o
filepath/to_tensor_test.cpp: In function ‘int main(int, char**)’:
filepath/to_tensor_test.cpp:12:48: error: no matching function for call to ‘from_blob(std::vector<double, std::allocator<double> >&, <brace-enclosed initializer list>, c10::TensorOptions&)’
     torch::from_blob(wavData, {1, len}, options);
                                                ^
In file included from filepath/libtorch/include/torch/csrc/api/include/torch/types.h:7:0,
                 from filepath/libtorch/include/torch/script.h:3,
                 from filepath/to_tensor_test.cpp:1:
filepath/libtorch/include/torch/csrc/autograd/generated/variable_factories.h:57:19: note: candidate: at::Tensor torch::from_blob(void*, c10::IntArrayRef, c10::IntArrayRef, const Deleter&, const c10::TensorOptions&)
 inline at::Tensor from_blob(
                   ^~~~~~~~~
filepath/libtorch/include/torch/csrc/autograd/generated/variable_factories.h:57:19: note:   candidate expects 5 arguments, 3 provided
filepath/libtorch/include/torch/csrc/autograd/generated/variable_factories.h:76:19: note: candidate: at::Tensor torch::from_blob(void*, c10::IntArrayRef, c10::IntArrayRef, const c10::TensorOptions&)
 inline at::Tensor from_blob(
                   ^~~~~~~~~
filepath/libtorch/include/torch/csrc/autograd/generated/variable_factories.h:76:19: note:   no known conversion for argument 1 from ‘std::vector<double, std::allocator<double> >’ to ‘void*’
filepath/libtorch/include/torch/csrc/autograd/generated/variable_factories.h:95:19: note: candidate: at::Tensor torch::from_blob(void*, c10::IntArrayRef, const Deleter&, const c10::TensorOptions&)
 inline at::Tensor from_blob(
                   ^~~~~~~~~
filepath/libtorch/include/torch/csrc/autograd/generated/variable_factories.h:95:19: note:   no known conversion for argument 1 from ‘std::vector<double, std::allocator<double> >’ to ‘void*’
filepath/libtorch/include/torch/csrc/autograd/generated/variable_factories.h:112:19: note: candidate: at::Tensor torch::from_blob(void*, c10::IntArrayRef, const c10::TensorOptions&)
 inline at::Tensor from_blob(
                   ^~~~~~~~~
filepath/libtorch/include/torch/csrc/autograd/generated/variable_factories.h:112:19: note:   no known conversion for argument 1 from ‘std::vector<double, std::allocator<double> >’ to ‘void*’
CMakeFiles/to_tensor_test.dir/build.make:82: recipe for target 'CMakeFiles/to_tensor_test.dir/to_tensor_test.cpp.o' failed
make[3]: *** [CMakeFiles/to_tensor_test.dir/to_tensor_test.cpp.o] Error 1
CMakeFiles/Makefile2:95: recipe for target 'CMakeFiles/to_tensor_test.dir/all' failed
make[2]: *** [CMakeFiles/to_tensor_test.dir/all] Error 2
CMakeFiles/Makefile2:102: recipe for target 'CMakeFiles/to_tensor_test.dir/rule' failed
make[1]: *** [CMakeFiles/to_tensor_test.dir/rule] Error 2
Makefile:138: recipe for target 'to_tensor_test' failed
make: *** [to_tensor_test] Error 2

CLion also puts a red line below torch::from_blob with the following:

No matching function for call to ‘from_blob’. candidate function not viable: requires at least 4 arguments, but 3 were provided

Note that I’m using C++ 17 (cannot use 14)