Using torch::from_blob(...) with const data

C++
1

Hello,
due to infrastructural limitations, I need to use a const input to from_blob to create a torch::tensor. Minimal reproducible example:

#include <iostream>
#include <torch/torch.h>

int main ()
{
    const float array[] = {1.23, 2.34, 3.45, 4.56, 5.67};
    auto options = torch::TensorOptions().dtype(torch::kFloat32);
    torch::Tensor t1 = torch::from_blob(array, {5}, options);
    std::cout << t1 << std::endl;
    return 0;
}

This raises however a compilation error: invalid conversion from ‘const void*’ to ‘void*’ [-fpermissive]. Of course I can cast away the const-ness of the array, but is there a way to avoid this? Can I somehow use from_blob (or similar) with const data input?

As always, help is highly appreciated… :slight_smile:
Thank you & Best regards,
RB

2

You can use std::copy_n as below:

const float array[] = {1.23, 2.34, 3.45, 4.56, 5.67};
auto options = torch::TensorOptions().dtype(torch::kFloat32);
torch::Tensor t1 = torch::empty({5}, options);
std::copy_n(array, 5, t1.contiguous().data_ptr<float>());
3

Hi @hoyden, thank you very much for your suggestion. Yes, you are right. I did a similar workaround with a simple memcpy(). The only issue is, that such a a memory copy per input and output tensor hurts runtime execution performance. I will apply this workaround, if there is no other way to deal with const arrays. But is there? :slight_smile:

Thank you very much & Best regards,
RB

4

I only know this one way to solve your problem. BTW, why you could only use a const input?

5

Hi @hoyden, thanks once again for your answer. It’s a bit tricky. We need to extend the ONNX Runtime (ORT) with operators that are not yet present. This can be done since ORT has a pretty nice C/C++ extension mechanism.

To save development time/effort we are basically using Libtorch in the ORT custom operators, and thus just need to interface the input/output data formats between ORT->Torch->ORT. However, the inputs, as defined by ORT come always as const float*. We don’t have a saying in this. We can always cast away the constedness, but that is also not the nicest thing…