Why reserved_memory+allocated memory is bigger than the memory shown in nvidia-smi?

It seems like that reserved_memory+allocated memory is bigger than the memory shown in nvidia-smi. My allocated mem is 3199MB and my reserved mem is 8766 MB, which equals 11965MB, but the mem shown in my nvidia-smi is 11131MB.

By the way, there is another question, I met an OOM when trying to allocate about 1000MB space but there was still about 8766MB in reserved space. The reserved space is designed to be reused, why is there still an OOM error?

The reserved memory reports the allocated memory and the cached memory so you shouldn’t sum them together. Take a look at the memory management docs for more information.

@ptrblck Thank you for your reply!
I have checked the docs. But I still have some confusion over the memory. I met an OOM error. RuntimeError: CUDA out of memory. Tried to allocate 1.53 GiB (GPU 0; 11.75 GiB total capacity; 5.97 GiB already allocated; 449.69 MiB free; 8.99 GiB reserved in total by PyTorch).
It seemed that I still have about 3G (8.99GiB-5.97GiB) cached memory to allocate tensors, but the allocation failed. So I guess 1.53 GiB tensors might be moved to free memory and then removed to cache memory, or the cached memory was occupied by Pytorch internal context management but why could it occupy so large space.
Did I make anything wrong? Thanks!

The error message points out that 449.69MiB are only free and the rest might be scattered and lost due to e.g. memory fragmentation.
The CUDA context is not captured by the allocated or reserved memory, as PyTorch doesn’t check its memory usage.