CUDA out of memory(specific question)

Sometimes this has to do with memory fragmentation. Sometimes you are just out of memory. Here clearly it’s the latter since 16MB > 320KB.

Typically users are puzzled about the memory fragmentation scenario where the OOM happens trying to allocate say 16MB, while reporting 200MB free, which makes no sense immediately, until you understand that in those 200MB there is not a single contiguous chunk of free memory that is larger than 16MB.

You can find the memory fragmentation discussed in multiple threads here on this forum, e.g. here is one of them where I was trying to figure it out: