How can I update the sampler and batch size dynamially

1

I am using WeightedRandomSampler to generate the weights for different labels.
And I want to change the weights gradually for each epoch.

But when I assign the new sampler back to Dataloader, it is not working.

The same story as for the batch size. Change the batch size after certain epoch.

thanks!

2 Likes
2

Just read the class for the dataloader, need to update the batch sampler as well. e.g

batch_sampler = torch.utils.data.sampler.BatchSampler(sampler, batch_size, False)
dataloders.sampler = sampler
dataloders.batch_sampler = batch_sampler

1 Like
3

Correct me if I am wrong, but doesn’t this code seem unnecessarily bulky ? Defining 3 classes for sampling seems to be an overkill: sampler, batch_sampler, and dataLoader.

I am not sure if the api allows it but can’t we sample from the loader using something as simple as: loader.get(indices) ? is there a performance gain from creating the above 3 objects ?

4

Well, update the

dataloders.sampler = sampler

is not necessary. Just to keep the sampler the same as batch_sampler.

I cannot find any elegant way to update the sampler and batch size, but the above way is not too bad ( 3 lines of code)

As the distribution of my training classes is unbalanced. Start sampling equally then gradually converge to final distribution seems working fine.

5

Thanks.

But how easy it is to sample using something like:
np.random.choice(4, p=(0.1,0.3,0.2,0.4))
with this framework ?

where p defines the sampling probability distribution.