Setting weight to the NLLLoss for imbalanced data

[tyron (Liu)]

I have 2 imbalanced classes with the size 3089 and 6179. How I can to weight the loss function in order to penalize errors on the smaller class. I am using nn.NLLLoss()

[Aman_Singh]

WeightedRandomSampler might be helpful to you.
Here’s a discussion about it.

Balanced Sampling between classes with torchvision DataLoader

Ok, I found some other issues. I changed your code a bit. This should work: numDataPoints = 1000 data_dim = 5 bs = 100 data = torch.FloatTensor(numDataPoints, data_dim) target = np.hstack((np.zeros(int(numDataPoints * 0.9), dtype=np.int32), np.ones(int(numDataPoints * 0.1), dtype=np.int32))) dataset_x = data.numpy() dataset_y = target from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(dataset_x, …

[KFrank]

Hi Liu!

@tyron

How I can to weight the loss function in order to penalize errors on the smaller class. I am using nn.NLLLoss()

You can use the weight argument that is passed to NLLLoss’s
constructor.

As an aside, if you do, in fact, have only two classes, you could
treat your problem as a binary classification problem (rather than
a general multi-class problem that happens to have two classes),
use BCEWithLogitsLoss, and use its pos_weight argument.

(Aman’s WeightedRandomSampler suggestion is also reasonable.
It and adding weights to the loss function are two somewhat different
approaches for compensating for imbalanced data.)

Best.

K. Frank