Multiple forward before backward, where backward depends on all forward calls

autograd
10

I noticed something off today with similar problem. Perhaps the root of the problem is the same to your case: Calling the forward function multiple times is not well-behaved.