Unable to clear CUDA cache

nlp
1

Hi,

I am trying to clear the CUDA cache using torch.cuda.empty_cache() but it is unable to clear it.

Following are the code and the error:

image
image901×912 92.4 KB

image
image929×518 95.6 KB

Thanks!

2

empty_cache will only release the unused and cached memory.
If the parameters and activations need more memory, you will run into an OOM error nevertheless.
You could try to lower the batch size or alternatively use torch.utils.checkpoint to trade compute for memory.