BCEWithLogitsLoss calculating pos_weight

I have a neural network as below for binary prediction. My classes are heavily imbalanced and class 1 occurs only 2% of times. Showing last few layers only

self.batch_norm2 = nn.BatchNorm1d(num_filters)

self.fc2 = nn.Linear(np.sum(num_filters), fc2_neurons)

self.batch_norm3 = nn.BatchNorm1d(fc2_neurons)

self.fc3 = nn.Linear(fc2_neurons, 1)

My loss is as below. Is this a correct way to calculate pos_weight parameter? I looked into official documentation at [this] and it shows that pos_weight needs to have one value for each class for multiclass classification. Not sure if for the binary class it is a difference scenario. I tried to input 2 values and I was getting error
(BCEWithLogitsLoss — PyTorch 1.11.0 documentation)

BCE_With_LogitsLoss=nn.BCEWithLogitsLoss(pos_weight=class_wts[1]/class_wts[0])

BCE is designed for binary classification, but it can be used in multi-label not multi-class classification.
what is the error?
only thing left is to have target with same shape of the output in one-hot format.

i am not using bce. I am using BCE_With_LogitsLoss for binary classification. Please confirm that my pos_weight calculation is correct

1 Like

Hi Ni!

The question is what are class_wts[1] and class_wts[0]?

A common choice for pos_weight would be
num_negative_samples / num_positive_samples. (This is
because you want to reweight the positive samples so that they
have about the same influence as the negative samples.)

If class_wts[1] were num_positive_samples (or proportional
to it), your pos_weight would be upside down.

Anyway, if “class-1” occurs about 2% of the time, you would want
pos_weight = (1.0 - 0.02) / 0.02 (which is 49.0 and is almost
equal to 50.0).

Best.

K. Frank

that makes sense. I need to switch pos_weight. My confusion was whether I need one value per class as shown at the official documentation. But that doesnt seem to be the case at least for binary prediction…