Error in the KL loss of pytorch

the formula used is

l_n = y_n * (log(y_n) - x_n) 

while it should be

l_n = y_n * (log(y_n) - log(x_n)) 

this could be solved with

loss(log(x), y)

instead of

loss(x, y)

But this is not pretty …

Did you do it on purpose and if you did why?

As it says on the page

the input given is expected to contain log-probabilities and is not restricted to a 2D Tensor. The targets are given as probabilities (i.e. without taking the logarithm).

So it’s not an error.

What happens is that starting from probs and then taking the logarithm is less numerically stable than directly taking the log as an argument.
Also, it’s the same convention as NLLLoss.

Best regards