loss = nn.MSELoss()
input = torch.randn(3, 5, requires_grad=True)
target = torch.randn(3, 5)
output = loss(input, target)
output.backward()
If I do this? output = loss(target, input)
Swap target and input.
Will this be a mistake?
I would say that it depends on the loss: for the MSE loss it is probably the same because mse(a,b) = mse(b,a), as you are doing the square of the error. The same holds for the mae criterion, because you are taking the absolute value. But be careful: this does not hold for measures such as the kullback leibler divergence, for which kll(a,b) != kll(b,a).
1 Like