CUDA runtime error out of memory while trying to build the model

I’m trying to build up a simple NN model and use GPU to run it. Then I have the memory error as trying to upload the model to Cuda. I have tried it locally on my computer, uploaded my code and run it on Kaggle notebook as well as Google colab, and the same issue happens for all the cases. So that I’m sure it’s caused by reasons other than PyTorch version, Cuda version, running environment, or batch size (since it’s not caused by allocating the input into GPU)

the code for building up the model is:
device = torch.device(‘cuda’)
des = model.DenseNet(num_init_features=32,num_classes=10)
cnn = nn.Sequential(
nn.Conv2d(1,3,5,1,2,bias=True),
des,
)
torch.cuda.empty_cache()
cnn.to(device)

The error detail on Google colab is like:

RuntimeError Traceback (most recent call last)
in ()
6 )
7 torch.cuda.empty_cache()
----> 8 cnn.to(device)
9

5 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in convert(t)
422
423 def convert(t):
–> 424 return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
425
426 return self._apply(convert)

RuntimeError: CUDA out of memory. Tried to allocate 2.00 MiB (GPU 0; 11.17 GiB total capacity; 10.52 GiB already allocated; 1.81 MiB free; 349.51 MiB cached

So as it shows it’s trying to allocate 2MB from 350MB space and failed, restarting kernel isn’t helping, using empty cache right in front of the code isn’t helping, everything is installed properly on local compute and the online running environment should be all set. I’ve searched almost all the cases from the internet and still can’t find the reason it doesn’t work, please help me.

did you go to runtime -> manage sessions -> get a list of active sessions, and terminate ones which you are not using in colab?

1 Like

I did and there’s only one active session there.

I was having a similar issue and this fixed it for me in google colab! thanks!