Saved intermediate values of the graph are freed when you call .backward() or autograd.grad()

[Salman_Abbasi]

In case someone is interested. The following helped me:

Detach, no_grad and requires_grad

Hello It’s general question but currently I’m looking at tutorial: http://pytorch.org/tutorials/intermediate/reinforcement_q_learning.html why in optimize_model function - we have detach() method and in select_action we use with torch.no_grad(): on the other hand doc: http://pytorch.org/docs/stable/notes/autograd.html mentions only requires_grad of course I understand we don’t want to compute gradients here - but i don’t fully understand the difference between all those 3 methods… als…

I had to detach and preferably empty all tensors used inside the for loops. I am not adding the code here. But for the for loops inside epoch loop, I have detached and zeroed tensors, before they are being computed and saved