Multi-class cross entropy loss and softmax in pytorch

No, F.softmax should not be added before nn.CrossEntropyLoss.
I’ll take a look at the thread and edit the answer if possible, as this might be a careless mistake!
Thanks for pointing this out.

EDIT: Indeed the example code had a F.softmax applied on the logits, although not explicitly mentioned.
To sum it up: nn.CrossEntropyLoss applies F.log_softmax and nn.NLLLoss internally on your input, so you should pass the raw logits to it.

12 Likes