Const correctness and thread-safety of torch::from_blob with external memory

1

In some C++ integrations, I often need to wrap externally managed buffers into torch::Tensor objects without copying data so they can be used for inference or transformations directly on CPU/GPU. The typical pattern uses:

torch::Tensor t = torch::from_blob(void* data, IntArrayRef sizes, const TensorOptions& options);

However, from_blob only accepts a non-const pointer, which means we must cast away constness even when the memory is immutable:

auto options = torch::TensorOptions().dtype(dtype).device(device).pinned_memory(true);
auto t = torch::from_blob(const_cast<void*>(block.ptr()), block.size(), block.stride(), options);

Const correctness

There’s currently no way to express that a tensor’s data should be read-only.
This forces all uses of external memory to go through a const_cast, which is error-prone and semantically incorrect when the source data must not be modified.

Even if the user only performs read-only operations, the compiler cannot enforce this, and developers must rely on assumptions about PyTorch internals.

Thread-safety

Frameworks used at CERN, like CMSSW, process data concurrently across threads.
If multiple threads access the same external buffer (declared const) via from_blob, we need strong guarantees that PyTorch will never write to that memory, even for internal bookkeeping or temporary aliasing. And a temporal copy operation would not be the solution where performance is crucial.

Without const correctness, it’s hard to ensure the use of from_blob remains safe in multi-threaded or read-only contexts.

Questions:

  1. Is the current void* signature of from_blob purely historical, or is there an internal reason preventing a const void* overload? I can see this would need internal effort in the framework from make tensor data const correct · Issue #97856 · pytorch/pytorch · GitHub, but there is no activity since 202,4 and I would not expect it to converge any time soon
  2. Can PyTorch guarantee that the memory passed to from_blob is it not written to unless an explicit in-place operation is invoked?
  3. Would the team consider adding support for a read-only tensor. If not possible short-term, is there at least a guarantee or documentation that read-only use (no mutations) of such tensors is safe and thread-safe?

Thanks in advance.

Regards,
Lukasz

2 Likes