Iterative Loss: Loss from first iteration when added in the second iteration does not work

[ptrblck]

@mbsyed

But when I update the model and try to do a backprop for the second iteration whose loss is the added loss of iteration 1 and iteration 2 even with retain_graph=True I get an error. I am not sure how to get around this and what is causing this issue.

I guess you are seeing an error, since you are trying to calculate the gradients (in iter2) from stale forward activations (calculated in iter1).
Have a look at this post for more information and check, if you are hitting the same error.