Extremely Imbalanced Data

data
1

Hi, everyone. I’m struggling with a very challenging issue. I have a binary database of 500K images. Of those 500K, only 10K are from one class and the rest are from a different class. I can’t use synthetic data.

I’m using a weighted loss, but that doesn’t change the fact that I’ll have more than 7800 steps in an epoch, considering batch=64. I was wondering if it makes sense to combine a Weighted Loss + sampler setting a fixed number of samples, or even Weighted loss + Weighted sampler.
Does that make sense? Is it conceptually correct?

The augmentations and loss I’m using now:

        aug = (
            [
                T.RandomHorizontalFlip(),
                T.RandomVerticalFlip(),
                T.RandomResizedCrop(img_size, scale=(0.75, 1.0)),
                T.RandomRotation(45),
                T.ColorJitter(hue=0.2),
            ]


#Loss
class_weights = class_weights.to(device)
self.loss_fn = lambda logits, labels: F.cross_entropy(logits, labels, weight=class_weights)
2

If you are working on a binary classification problem, having 10K images per class is generally sufficient for training. A straightforward approach is to balance the dataset, since you already have enough samples for both classes.

You can achieve this by randomly sampling 10K images from the remaining 490K images of the other classes. This will result in a balanced dataset with a total of 20K images.

3

Thanks for your reply. But it seems that the model isn’t learning anything. 490k samples from one class vs 10k from another class is too much imbalance.
the results show a super overfited model

4

Or are you saying I should only do the augmentations in the minority class?

5

Assuming you have 10K images for class-0 and 490K images for class-1, I suggest selecting 10K samples from the 490K available for class-1. This will give you a balanced dataset with 10K samples for each class.

You can then train your model on this new, balanced dataset. You won’t need to use the class_weights as the dataset is balanced.

6

Thats a good idea, thanks. But I have another dataset with 0 - 100899 and 1 - 54.

What do I do in that case? The first one I agree with you, I think its a good approach but the other one, the version with 54 images is killing me.

I cannot combine this set with others. ):

7

There is no mention of the domain you are working in. You might consider using pre-trained models (typically trained on ImageNet-1K), which require far fewer samples (often, even 100 can be sufficient) for convergence. You can also augment class-1 to increase the number of samples to at least 100.

8

I am working with medical images, with different devices and diagnoses. And according to the rules of the problem, I cannot mix them