When my input and target have the shape (3, 5), the outputs of SmoothL1Loss(reduction='none') and MSELoss(reduction='none') have the shape (3, 5). What does this mean? It has to have the shape (3, 1). That means it has to calculate one value per input-target pair.
when you use reduction='none', it will compute the MSE-loss per element. So you can add an additional torch.sum() or torch.mean() after the output of MSE-loss. Alternatively, you can use reduction='sum' or reduction='mean' to the same thing.
However, using reduction='none' gives you more flexibility. For example, if you want to calculate the sum over one dimension but the calculate the mean of the other dimension, you can use loss=MSELoss(reduction='none') followed by loss=torch.mean(loss, dim=1) and then loss=torch.sum(loss, dim=0)
1 Like