Loss evaluation

1

I am trying to evaluate the loss for my training and validation set. But I am not sure what is loss evaluated through loss.backward(). ie. if the loss function is as cross-entropy. Like does it return the averaged loss per batch? So that means I should have times the batch size in order to get the total loss? Or looking at the loss per batch is sufficient enough?

2

From the docs:

  • reduction (str, optional) – Specifies the reduction to apply to the output: 'none' | 'mean' | 'sum'. 'none': no reduction will be applied, 'mean': the weighted mean of the output is taken, 'sum': the output will be summed. Note: size_average and reduce are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction. Default: 'mean'

So by default 'mean' will be used.