I am currently using the torch.Tensor.backward(retain_graph=True, create_graph=True)
method and I have a memory leak.
According to the docs, this can be expected, and in order to avoid this, the .grad
field of the parameters should be reset. I am performing this with the following snippet:
for param in model.parameters():
param.grad = None
however, I am still getting the memory leak. Am I missing something to be reset?
Note that I cannot use the alternative torch.autograd.grad()
function, because I have registered some backward hooks, that would not otherwise be called.