I am training a binary classifier, however I have a softmax layer as the last layer, thus is it ok if I use nn.CrossEntropyLoss() as objective function instead of Binary Cross entropy loss? are there any difference?
I tried using nn.CrossEntropyLoss()
for binary classification and it didn’t work. You should use BCELoss
.
What are the other loss functions which work for multi-class classification(one is Cross entropy.)?
MultiLabelMarginLoss, MultiLabelSoftMarginLoss, MultiMarginLoss
Shouldn’t nn.CrossEntropyLoss()
be equivalent to nn.Sigmoid() --> nn.BCELoss()
? (With two / one output respectivly).
But BCE Loss does Sigmoid, I have a softmax layer in the end, any workaround?
BCE Loss does not do sigmoid, you can look at the doc http://pytorch.org/docs/master/nn.html#torch.nn.BCELoss
If you want BCE to do sigmoid, use BCE with logits http://pytorch.org/docs/master/nn.html#torch.nn.BCEWithLogitsLoss
Yes exactly. You might want to follow this little thread on SO to get this clearer: https://stackoverflow.com/questions/53628622/loss-function-its-inputs-for-binary-classification-pytorch
nn.CrossEntropyLoss
for binary classification didn’t work for me too! In fact, it did the opposite of learning. Why didn’t it work for you? Can you please explain the behavior I am observing?
Note: The same model with nn.MSELoss
worked fine (passed overfitting test). And I am supplying the unnormalized FC layer output to the CE loss function.