I’ve looked around online but I still haven’t been able to figure out how to properly free GPU memory, here’s a link to a simple Colab demo explaining the situation [make sure to change the Runtime type to use a GPU]:
I basically start by allocating a random tensor, move it to the GPU, report the GPU memory usage, then move the tensor back to the CPU, report the GPU memory usage, then delete the reference to the tensor and report the GPU memory usage once more. On the last two memory reports I’d expect the usage to go back to zero, but I keep getting a non zero answer (e.g. the usage goes down from 37% to 4%, but not zero).
How can I get the GPU memory back to zero in this simple case? Is this an issue with Google Colab specifically?
Thanks in advance!