OutOfMemoryError

1

When running a model, I got the following error message

OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 4.00 GiB total capacity; 3.29 GiB already allocated; 0 bytes free; 3.44 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

How to fix this error, i.e., what should be the right setting for

max_split_size_mb

and

PYTORCH_CUDA_ALLOC_CONF

To debug this issue, are there any GPU-related information I can provide?

2

I would recommend trying to reduce the memory usage by e.g. lowering the batch size given the GPU memory is quite small with 4GB total memory.