You can see there are two different loss calculation methods in following code snippet. The first approach is quite common and easy to understand, while the second one is a bit obscure to me. Can someone shed any lights on how to interpret this? Much appreciated.
for batch_index, batch_dict in enumerate(batch_generator):
........
y_pred = self.model(batch_dict['train'])
loss = self.loss_func(y_pred, target)
## Approach One:
running_loss += loss.item()
## Approach Two:
running_loss += (loss.item() - running_loss) / (batch_index + 1)
........