I’m trying to send my CNN model to the GPU device, but each time I run model = model.to(device) I got an error “RuntimeError: CUDA error: out of memory”.
I tried to use
import torch
torch.cuda.empty_cache()
but that did not work, I’ve restarted the Kernal but that didn’t solve the problem. I checked the free/used memory, it looks full, I’ve tried to clean the memory using torch.cuda.empty_cache() that did not work, the below image shows the free/used memory.
I don’t have any idea why this error pops-up even I don’t send or train any model on the GPU.
Try these following steps to figure out where the problem is:
Use the model with a simple most basic training loop. If the problem is solved check your training loop for any accumulation or (maybe make sure you’re not calling model.to(device) for each epoch)
If the error persists try using a different model. To find out if the error is in your model. (Has happened to me before and was due to some linear layers)
Simple training loop: (no autograd or scalers)
def train_loop(dataloader, model, loss_fn, optimizer):
size = len(dataloader.dataset)
for batch, (X, y) in enumerate(dataloader):
pred = model(X)
loss = loss_fn(pred, y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
if batch%100 ==0:
loss, current = loss.item(), batch*len(X)
print(f"loss: {loss:>7f} [{current:>5d}/{size:>5d}]")
epochs = 10
for t in range(epochs):
print(f"Epoch {t+1}\n-------------------------------")
train_loop(train_dataloader, model, loss_fn, optimizer)
print("Done!")
