About BCEWithLogitsLoss's pos_weights


(Wang Tan) #1

Hello everyone, I’m a new in pytorch and it’s quite great.But I have some problem when using the BCELoss and I have tried to find answer in the forum but it doen’t work. So I have to creat a topic.

Now I am solving a problem about the multilabel classification and use the BCEWithLogitsLoss ( just define learning rate). I have 500 label, each label can be 0 or 1, and 0 is much more than 1.(Only in first 2 label, 0 and 1 are approximatly equall).

When I try to train it. First, the loss is about 0.69, and it decreace quickly to 0.1 in 1 epoch. And then loss basically do not fall. And i try to print the output (before the sigmoid) and the label. Find that all the value are negative.
image
image

I feel very strange. And I just guess if it’s because that in label 0 is much more than 1? And the output almost are negative. If it is, whether i should use the pos_weight?


#2

If your classes are imbalanced, setting the pos_weigth to number_neg_samples / number_pos_samples should give a more diverse prediction.


How to apply weighted loss to a binary segmentation problem?
(Wang Tan) #3

Thankyou very much for reply.
I have a try but a little worried, because the pos_weight on some class can be 100, more than the three in the docs.
Does it matter?

Best regards


#4

You would have to check your per-class accuracies or the confusion matrix.
The weighting will trade one accuracy for another, so it’s a way to balance your predictions.
Currently your model would have a perfect true negative rate, while the true positive rate is most likely low.


Dealing with imbalanced datasets in pytorch
(Xiao Li) #5

hello, i am doing a task of multilabel classification, may i have your wechat to chat with you about the problem of unbalanced data? Thank you, i am master candidate.