Memory use on list comprehension

autograd
1

Hi,

I am having some issues with a list of tensors:

My list is
features = [tensor(batch, dim1, dim2, etc.), ..., tensor(batch, dim1, dim2, etc.)]

I want to restrain it to some elements, so I do

keep = elements
features = [f[keep] for f in features]

I would expect same or less memory to be used but instead I see a spike in memory use. I am guessing there is a copy but I am not sure how to avoid it while still being able to save a list of tensors.

Edit: if it is relevant, this is on Pytorch 1.0 and it is GPU memory which spikes. I do other operations afterwards/before but commenting this line removes the memory overhead.

2

I created a similar example, and measure the allocated memory on the GPU, but before and after this list comprehension, there was no change in allocated and cached memories:

>>> import torch
>>> device = torch.device('cuda:0')
>>> torch.cuda.memory_allocated()
0
>>> torch.cuda.memory_cached()
0
>>> t = [torch.randn(100, 100).to(device) for i in range(10)]
>>> torch.cuda.memory_allocated()
404480
>>> torch.cuda.memory_cached()
1048576
>>> 
>>> 
>>> t = [v[0] for v in t]
>>> torch.cuda.memory_allocated()
404480
>>> torch.cuda.memory_cached()
1048576
3

Hmm, weird, must be an interaction with the rest of my programme. Anyway, I restructured some of the code to work around the list comprehension.
Although in your case you would expect 1/100 of the memory to be allocated no?

4

There was another post on « freeing GPU memory » : How to delete PyTorch objects correctly from memory

It does not show that the allocated memory has changed, but if I create a new tensor, it will reuse the freed memory.