Unfortunately, this solution still doesn’t work for me :( I see memory_allocated increasing batch after batch while the tensors returned by this function don’t change… any thoughts?
I guess based on @smth’s comment, there is something in the graph that’s being kept around but not reported by gc… I am having a memory bug here (Advice on debugging a GPU memory leak in graph?) where, even when the model is in eval mode and with torch.no_grad(), there is increasing memory. However, I tried creating a minimal working example that creates a node in the graph (via a matmul) by multiplying an input by a parameter that requires a gradient, and then calls a forward pass many times, but I don’t see any increase in allocated memory.
For what it’s worth, in my other post, if I replace the matmul with a simple +, there’s no memory leak…