Adagrad cuda runtime error (2)

Thought I’d share for anyone else having this issue:

I was getting the following error when I was trying to run a GAN architecture with adagrad optimizer on the generator.

THCudaCheck FAIL file=/pytorch/torch/lib/THC/generic/ line=58 error=2 : out of memory
Traceback (most recent call last):
File “”, line 29, in
g_optim = torch.optim.Adagrad(generator.parameters(), lr=lr)
File “/imatge/lpanagiotis/ENV/local/lib/python2.7/site-packages/torch/optim/”, line 30, in init
state[‘sum’] = torch.zeros_like(
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/torch/lib/THC/generic/
srun: error: c8: task 0: Exited with exit code 1

Switching to SGD instead fixed the error and the code run normally. I am bamboozled as to why would Adagrad give such an error, and I would be interested to hear others’ thoughts on it.

I am not an Adagrad expert but from what I remember, it keeps track of extra information for every parameter to perform it’s update. So it’s memory usage is larger than SGD.

1 Like