I am having some issues with a list of tensors:
My list is
features = [tensor(batch, dim1, dim2, etc.), ..., tensor(batch, dim1, dim2, etc.)]
I want to restrain it to some elements, so I do
keep = elements
features = [f[keep] for f in features]
I would expect same or less memory to be used but instead I see a spike in memory use. I am guessing there is a copy but I am not sure how to avoid it while still being able to save a list of tensors.
Edit: if it is relevant, this is on Pytorch 1.0 and it is GPU memory which spikes. I do other operations afterwards/before but commenting this line removes the memory overhead.
I created a similar example, and measure the allocated memory on the GPU, but before and after this list comprehension, there was no change in allocated and cached memories:
>>> import torch
>>> device = torch.device('cuda:0')
>>> t = [torch.randn(100, 100).to(device) for i in range(10)]
>>> t = [v for v in t]
Hmm, weird, must be an interaction with the rest of my programme. Anyway, I restructured some of the code to work around the list comprehension.
Although in your case you would expect 1/100 of the memory to be allocated no?
There was another post on « freeing GPU memory » : How to delete PyTorch objects correctly from memory
It does not show that the allocated memory has changed, but if I create a new tensor, it will reuse the freed memory.