Tensor type memory usage

Memory Format
1

I would like to know how much memory (on GPU) do the different torch
types allocate, but I was not able to find it anywhere in the docs.

At the following link all the possible Tensor types are specified, but nothing is said about memory usage.
https://pytorch.org/docs/stable/tensors.html

For example, i suppose that the following two tensors have different memory usage, but is there something I can print out ?

a = torch.ones([10000], dtype=torch.uint8, device='cuda:0')
b = torch.ones([10000], dtype=torch.float32, device='cuda:0')

I see that in the protected attributes there is “_cdata”, what is it ?
I initially thought that it was the attribute I was looking for, but it can’t be, since:

a = torch.ones([10000], dtype=torch.uint8, device='cuda:0')
b = torch.ones([10000], dtype=torch.uint8, device='cuda:0')

a._cdata
b._cdata

prints out:

94394199987296
94392001480192

2

With Storage you can get some information about what is happening.

For example data_ptr() will give you the address of the first element.

a = torch.ones([10000], dtype=torch.uint8, device='cuda:0')
b = torch.ones([10000], dtype=torch.float32, device='cuda:0')

print(a.storage().data_ptr())
print(b.storage().data_ptr())

# 140208108520960
# 140208108531200

In the documentation linked above you can see all the information that you can get from there, such as

  • get_device()
  • nbytes()
  • is_pinned()
  • …

Using nbytes() you can see the difference that you wanted to see between uint8 and float32

print(a.storage().nbytes())
print(b.storage().nbytes())

# 10000
# 40000

Hope this helps :smile:

3

Hi, thank you for your answer.

I tried with your suggestion, but it gives me and error:

a.storage().nbytes()

AttributeError: ‘torch.cuda.ByteStorage’ object has no attribute ‘nbytes’

But I found out that to get the number of bytes:

a = torch.ones([10000], dtype=torch.unit8, device='cuda:0')
b = torch.ones([10000], dtype=torch.float32, device='cuda:0')

a.element_size()
b.element_size()

1
4

Anyway, I could not find any information about what _cdata() represents ?

4

Oh yeah, nbytes() was intoduced on pytorch 1.11 I think. So if you want to use that you need to update your version with

!pip install --update torch
!pip install --update torchvision
# Or however you need to update it (if you want to do it like this, but you also got an alternative)

And if I find anything on _cdata() I will post it, but I also have no idea

1 Like