Unrealistic loss using BCEWithLogitsLoss()

I am trying to use nn.BCEWithLogitsLoss() with ResNet50 architecture for a binary classification task.
In order to do that, I have set the net’s fully connected layer in this way: fc = nn.Linear(2048, 1). Furthermore, I have also changed the targets’ shape: labels = labels.unsqueeze(1).float().
It seems don’t work properly, because using this kind of loss I get an accuracy score greater than 1, and a loss very near 0.

Afterward, I tried to use nn.CrossEntropyLoss() and fc = nn.Linear(2048, 2) without unsqueeze the labels; and it works correctly.

I would like to know why nn.BCEWithLogitsLoss() returns these strange values, and nn.CrossEntropyLoss() works properly instead.

Check the min./max. values in your target and make sure they are in the range [0, 1].
Also, how do you calculate the accuracy? Since your model outputs logits you should use a threshold e.g. via:

preds = output > 0.0
1 Like

when you want to calculate the accuracy after you obtained the prediction with model, you need to apply sigmoid as the outputs of your models are logitss (it can be from -inf to inf). then, apply thresholding after it.
basically, what BCEWithLogits does, is to apply sigmoid then calculates BCELoss. if you are calculating sigmoid by your own, you dont need BCEWithLogits. you instead want to use BCELoss
if you are using BCEWithLogits, after you obtain your predictions, you need to apply sigmoid to calculate accuracy and sigmoid should not be applied when passing model’s output to loss