[Solved] Why does a cuda float tensor with 64 million floats use ~512MB GPU?

[QuantScientist (Solomon K)]

ASAIK, PyTorch uses a caching allocator, while the memory is “free”, this is not reflected in the view from the device.

How to clear some GPU memory?

Even if that same process can reuse the GPU memory, it doesn’t look like other processes can. I’m running into a similar utilization concern. Another process will run into Out of Memory errors, while the original process keeps the GPU memory even after it is done using it.