CUDA out of memory, but why?

Memory Format
1

When I run my code, I get this runtime error below:

CUDA out of memory. Tried to allocate 88.00 MiB (GPU 0; 8.00 GiB total capacity; 6.04 GiB already allocated; 0 bytes free; 6.17 GiB reserved in total by PyTorch)

I don’t understand why it says 0 bytes free;
Maybe I should have at least 6.17 - 6.04 = 0.13 GiB free?

I’m afraid if it’s not really a memory shortage problem.

2

You are trying to allocate 88MB. ~130MB are in the cache, but are not a contiguous block, so cannot be used to store the needed 88MB. 0B are free, which shows that the rest of your device memory is used to store the CUDA context, memory for other applications etc.

1 Like