WeightedRandomSampler - Imbalanced classes Multiclass classification

When we use WeightedRandomSampler for a mutliclass problem, we set reduction of the loss to mean? (always?) and what this ‘mean’ means?

The usage of a WeightedRandomSampler is independent of the used loss function.
For e.g. nn.CrossEntropyLoss, the reduction='mean' uses

The losses are averaged across observations for each minibatch.

as explained in the docs.


Could you explain me briefly what’s the purpose of WeightedRandomSampler? I found an answer you gave here few years ago and I used it, everything ok but I don’t know how exactly I did for my imbalanced data.

Also is it ok that I used both WeightedRandomSampler with reduction = mean?

WeightedRandomSampler is used to provide weights to each sample, which is used during the sampling process of selecting the data samples for each batch.
The higher the weight assigned to a particular index, the more likely this data sample will be used in a batch.
To created batches of balanced classes, you would therefore assign a higher weight to samples from the minority class, and a lower weight to samples from the majority class.

Yes, the reduction is independent from the sampling strategy.

1 Like