I am training a binary classifier, however I have a softmax layer as the last layer, thus is it ok if I use nn.CrossEntropyLoss() as objective function instead of Binary Cross entropy loss? are there any difference?
9 Likes
I tried using nn.CrossEntropyLoss() for binary classification and it didn’t work. You should use BCELoss.
2 Likes
What are the other loss functions which work for multi-class classification(one is Cross entropy.)?
1 Like
MultiLabelMarginLoss, MultiLabelSoftMarginLoss, MultiMarginLoss
3 Likes
Shouldn’t nn.CrossEntropyLoss() be equivalent to nn.Sigmoid() --> nn.BCELoss()? (With two / one output respectivly).
3 Likes
But BCE Loss does Sigmoid, I have a softmax layer in the end, any workaround?
BCE Loss does not do sigmoid, you can look at the doc http://pytorch.org/docs/master/nn.html#torch.nn.BCELoss
If you want BCE to do sigmoid, use BCE with logits http://pytorch.org/docs/master/nn.html#torch.nn.BCEWithLogitsLoss
4 Likes
Yes exactly. You might want to follow this little thread on SO to get this clearer: https://stackoverflow.com/questions/53628622/loss-function-its-inputs-for-binary-classification-pytorch
nn.CrossEntropyLoss for binary classification didn’t work for me too! In fact, it did the opposite of learning. Why didn’t it work for you? Can you please explain the behavior I am observing?
Note: The same model with nn.MSELoss worked fine (passed overfitting test). And I am supplying the unnormalized FC layer output to the CE loss function.
1 Like