Getting tensor raw data

1

Hi,
Simple question:
I have a 2D tensor and I want to get its raw data, i.e the actual bytes in that tensor, regardless of what they represent.

How do I do it?

Thanks,
Guy

2

If t is your tensor, you could try t.numpy().tobytes(). Is that what you are looking for?

3

So I have a half-precision model(float16), and I want to avoid going through numpy, because I suspect there is some compatability issue between numpy and pytorch for float16. It may not be the case, but I want to verify it without going through numpy

4

Oh, I see. Then how about using t.data_ptr().to_bytes(length=16,byteorder=sys.byteorder)?

5

Hi,
Thanks for you answer!

it seems that data_ptr is returning an address, i.e an integer:
ipdb> self.rec_net.weight_hh_l0.data.cpu()
tensor([[ 5.4779e-02, 4.0552e-01, 2.6550e-03, …, -5.7739e-02,
8.5938e-02, -1.3538e-01],

    ...,
    [-1.4709e-01, -3.9398e-02,  6.1707e-02,  ..., -3.1787e-01,
      2.7271e-01,  2.5806e-01],
    [ 1.4595e-02,  2.8725e-03,  2.7222e-01,  ...,  1.6602e-01,
     -7.5928e-02, -2.1057e-01]], dtype=torch.float16)

ipdb> self.rec_net.weight_hh_l0.data.cpu().data_ptr()
140297234031872
ipdb> self.rec_net.weight_hh_l0.data.cpu().data_ptr().to_bytes(length=16,byteorder=sys.byteorder)
*** AttributeError: ‘int’ object has no attribute ‘to_bytes’

Did it work for you?

Thanks,
Guy

6

Yes, the following worked for me, using python 3.6 and pytorch 0.4:

t = torch.HalfTensor([[1.23,2.34],[3.45,4.56]])
t.data_ptr().to_bytes(4 * 16,sys.byteorder)

Output:  b' \xe9f\n\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

Although I am not sure wether that gives me the bytes of the array or the bytes of the address.