Why is "accumulating" the default mode of .gradient?

Why didn’t you guys just make it overwrite? Is there any specific reason for that?

One possible reason I can think of is one variable may contribute to multiple losses and in the backward pass, the gradients should be accumulated from all losses.

1 Like

Hi Aerin! Nice to see you here :slight_smile:

I found this post with an answer by @albanD - Why do we need to set the gradients manually to zero in pytorch?
It explains the decision to accumulate gradients when .backward() is performed. I assume the same argument applies for .gradient().

2 Likes