Backward() on the sum of loss array

autograd
1

I’m fixing the torch.manual_seed() and I was expecting the two code snippets below to lead to the same results, however calling backward() inside the loop seems to lead to a better performance. I’m struggling to understand whey there is a difference between the two, could this be a numerical problem (the sum of loss values overflows?)

backward() inside the loop:

        optimizer.zero_grad()
        for loss in episode_losses:
            weighted_loss = loss * reward
            weighted_loss.backward()
        optimizer.step()

backward() on the sum:

        optimizer.zero_grad()
        total_loss = 0
        for loss in episode_losses:
            weighted_loss = loss * reward
            total_loss += weighted_loss
        total_loss.backward()
        optimizer.step()
2

they both should be equivalent, both in terms of numerical stability and computation. I wonder if there are some non-deterministic operations you are using, and that’s the cause of the difference in performance.