I am trying to free the GPU memory, since i am not running any code and it is full.
I also closed the Visual Studio Code, and re-open it and nothing happens.
I also used torch.cuda.empty_cache() but the memory is not released.
I’m really not versed in reading nvidia-smi output but it looks to me like you have a bunch of python processes running that are using up GPU memory (bottom section of your image). Could you kill those and check if that helps release your memory? e.g. run $ kill 111715 in your shell, where 111715 is the PID of the first process that shows up in nvidia-smi.
From this section of nvidia-smi, the GPUs seem to be far from full:
From your error message…
…it seems like you’re trying to load a dataset of about 11.7 GB into memory, and each of your GPUs can only 6 GB available (and maybe your CPU also doesn’t have enough memory for this). Perhaps you could somehow split up the dataset and send half of it to one GPU and another half to another GPU (I’ve never done that before so I’m not sure how that works).
However, just wanted to say that it is atypical to load the entire dataset into memory. What’s more typical is to use dynamic / lazy loading, using Datasets and DataLoaders like here.
Thanks, I will look to it, but it is strange since like for 10 days I was able to upload that dataset on the CPU and the code did not break in that section.
Your CPU will have more or less memory available depending on what else is running on your system, independent of the GPU and what nvidia-smi might tell you. So it’s possible that it would work at some moments but not others, since the amount of memory consumed by other processes is not going to be consistent over time.