CUDA Out of Memory Error in Pytorch

I am trying to run my model on jupyter notebook using pytorch_geometric framework. But, I am facing this error. When I train my model, I am facing this error. Is there anyway to fix this error?

Error is this one:

OutOfMemoryError Traceback (most recent call last)
----> 1 model.forward(train_data[“feature”], train_data[“edge_index”], train_data[“edge_weight”] )

6 frames
/usr/local/lib/python3.8/dist-packages/torch_geometric/nn/conv/ in lift(self, src, edge_index, dim)
234 try:
235 index = edge_index[dim]
→ 236 return src.index_select(self.node_dim, index)
237 except (IndexError, RuntimeError) as e:
238 if index.min() < 0 or index.max() >= src.size(self.node_dim):

OutOfMemoryError: CUDA out of memory. Tried to allocate 25.04 GiB (GPU 0; 14.76 GiB total capacity; 1.73 GiB already allocated; 12.11 GiB free; 1.87 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

You could try to decrease the batch size the reduce the memory usage. Alternatively, torch.utils.checkpoint could also help but I don’t know how well PyG is supported.

1 Like