Hi all, before adding my model to the gpu I added the following code: def empty_cached(): gc.collect() torch.cuda.empty_cache() The idea buying that it will clear out to GPU of the previous model I was playing with. Here’s a scenario, I start training with a resnet18 and after a few epochs I …

You would have to delete all references to the tensors you would like to free. gc.collect() and empty_cache() will not free tensors, which are still used and referenced.

I see! How can I delete references to the tensors? I tried with del(model, optimizer, criterion) but when I query the GPU memory before and after it seems like its the same.

del tensor should work. To release the cached memory, you would need to call torch.cuda.empty_cache() afterwards. Here is a small example: print(torch.cuda.memory_allocated()/1024**2) print(torch.cuda.memory_cached()/1024**2) x = torch.randn(1024*1024).cuda() # 4MB allocation and potentially lar…

Amazing, it works great! I was using the nvidia-smi to determine the memory allocation on the GPU. What would be the difference between that and torch.cuda.memory_allocated.

nvidia-smi shows the all processes which use memory on the device. So besides PyTorch or course other processes might use memory. Also, note that PyTorch loads the CUDA kernels, cudnn, CUDA runtime etc. after the first CUDA operation, which will also allocate memory (and cannot be freed until the …

@ptrblck sir, I have the same problem here, can you help me determine where I put the cuda. empty_cache()? [image] RuntimeError: CUDA out of memory? vision Check the attention_maps and raw_features, as well. If they are not scalers, and they do not participate in the trai…

No, you cannot delete the CUDA context while the PyTorch process is still running and would have to shutdown the current process and use a new one for the downstream application.

Is there a conveninent method for deleting all the model tensors in one go?

Clearing the GPU is a headache

vision

ljj7975 (Brandon Lee) July 8, 2021, 3:04pm 9

is there a way to programmatically clear CUDA context?

I would like to release the memory allocated by CUDA context after model training because my process need to do other operations after that