Frombuffer() → "The given buffer is not writable"

Memory Format
1

¿Why when I run:

file = open('/tmp/big.bin', 'rb')
file_bytes = file.read()
mytensor = torch.frombuffer(file_bytes, dtype=torch.float32).reshape(a, b)

do I get this:

UserWarning: The given buffer is not writable, and PyTorch does not support non-writable tensors. This means you can write to the underlying (supposedly non-writable) buffer using the tensor. You may want to copy the buffer to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /tmp/SBo/pytorch-v2.3.1/torch/csrc/utils/tensor_new.cpp:1524.)

Why is the file buffer not writable? (It gives this warning regardless if I run file.close() before frombuffer() or not.) And what does the file buffer’s writability have to do with the tensor’s writability?

Also, mytensor has some nans in it. I don’t get a tensor with any nans when I use torch.uint8 as the dtype.

2

you’ve opened the file in rb mode which stands for read-only binary mode.

And what does the file buffer’s writability have to do with the tensor’s writability?

from_buffer uses Python’s buffer interface and attempts to directly map to the underlying buffer without copying the data. So, the tensor is directly mapping to the file, it is not separately materializing or copying the data from the buffer into new memory. So torch is warning you that attempts at changing the tensor values might result in a new tensor and those changes aren’t reflected in the file itself.

Also, mytensor has some nan s in it. I don’t get a tensor with any nan s when I use torch.uint8 as the dtype .

Sure, that depends on the contents of the file.