WeightedRandomSampler Not Random

1

Hi there,

I have an imbalanced dataset, and I am trying to make each batch contains every class, regardless of the numbers as long as one exists. I found WeightedRandomSampler may be working. However, after I assign weights, the first batch always only draws samples from class0.

There are six classes in total.

class_sample_counts = [568330.0, 430.0, 349.0, 209.0, 14590.0, 9712.0]
class_weights = 1./torch.tensor(class_sample_counts)

sampler = WeightedRandomSampler(weights=class_weights,num_samples= HDF5Dataset(path=hdf5_path,set_name="train").__len__(),replacement=True)

train_loader = DataLoader(HDF5Dataset(path=hdf5_path,set_name="train"), batch_size=config['batch_size'],drop_last=False,sampler=sampler)
val_loader = DataLoader(HDF5Dataset(path=hdf5_path,set_name="val"), batch_size=config['batch_size'],shuffle=True,drop_last=False)
test_loader = DataLoader(HDF5Dataset(path=hdf5_path,set_name="test"), batch_size=128,shuffle=True,drop_last=False)

loaders = {"train":train_loader,"val":val_loader,"test":test_loader}

a = iter(train_loader)
print((next(a)[1]))
2

Here is the output.

tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0.])
3

The weights should be passed as sample weights (a weight is assigned to each sample), not class weights.
Here is an example on how to use the WeightedRandomSampler.