About BCEWithLogitsLoss's pos_weights

Hello everyone, I’m a new in pytorch and it’s quite great.But I have some problem when using the BCELoss and I have tried to find answer in the forum but it doen’t work. So I have to creat a topic.

Now I am solving a problem about the multilabel classification and use the BCEWithLogitsLoss ( just define learning rate). I have 500 label, each label can be 0 or 1, and 0 is much more than 1.(Only in first 2 label, 0 and 1 are approximatly equall).

When I try to train it. First, the loss is about 0.69, and it decreace quickly to 0.1 in 1 epoch. And then loss basically do not fall. And i try to print the output (before the sigmoid) and the label. Find that all the value are negative.
image
image

I feel very strange. And I just guess if it’s because that in label 0 is much more than 1? And the output almost are negative. If it is, whether i should use the pos_weight?

1 Like

If your classes are imbalanced, setting the pos_weigth to number_neg_samples / number_pos_samples should give a more diverse prediction.

Thankyou very much for reply.
I have a try but a little worried, because the pos_weight on some class can be 100, more than the three in the docs.
Does it matter?

Best regards

You would have to check your per-class accuracies or the confusion matrix.
The weighting will trade one accuracy for another, so it’s a way to balance your predictions.
Currently your model would have a perfect true negative rate, while the true positive rate is most likely low.

hello, i am doing a task of multilabel classification, may i have your wechat to chat with you about the problem of unbalanced data? Thank you, i am master candidate.

Hi… We need to use pos_weight from only train_dataset or for entire dataset for a multilabel approach?