Why the min loss is not zero in neither of MultiLabelSoftMarginLoss and BCEWithLogitsLoss

[KFrank]

Hi Seyeeet!

@seyeeet

2. why the minimum of neither of those two losses is not zero when I give them the input equal to targets?

BCEWithLogitsLoss takes raw-score logits (that run from -inf to
inf) as its input, but takes probabilities (0.0 to 1.0) as its target.

Instead of using 0.0 and 1.0 for your input, use very large negative
and positive values. Then you will get a loss of (something close to)
0.0.

(In general, you shouldn’t use BCELoss, but if you apply it to your
example values, you will get 0.0 for the loss.)

loss_1 = BCEWithLogitsLoss(output,target) # loss_1 will be equal to 0.5032
loss_2 = MultiLabelSoftMarginLoss(output,target) # loss_2 will be equal to 0.5032

why they are even equal?

BCEWithLogitsLoss and MultiLabelSoftMarginLoss are essentially
the same. See, for example:

What is the difference between BCEWithLogitsLoss and MultiLabelSoftMarginLoss

I think there is no difference between BCEWithLogitsLoss and MultiLabelSoftMarginLoss. BCEWithLogitsLoss = One Sigmoid Layer + BCELoss (solved numerically unstable problem) MultiLabelSoftMargin’s fomula is also same with BCEWithLogitsLoss. One difference is BCEWithLogitsLoss has a ‘weight’ parameter, MultiLabelSoftMarginLoss no has) BCEWithLogitsLoss : [image] MultiLabelSoftMarginLoss : [image] The two formula is exactly the same except for the weight value.

Best.

K. Frank