DataLoader api confusion

1
  1. Why DataLoader has both sampler as well as batch_sampler? while there is already a sampler that is torch.utils.data.BatchSampler? does that mean I can pass BatchSampler only as batch_sampler of DataLaoder?
  2. If I want a DistributedSampler that is also in batches/BatchSampler and is random/RandomSampler I will have no option to mix them!
    both DistributedSampler and RandomSampler wrap a dataset/data_source and only BatchSampler can wrap one of them.
  3. Even if we could mix them, what would be difference of RandomSampler(BatchSampler(DistributedSampler(dataset))) and BatchSampler(RandomSampler(DistributedSampler(dataset))).
  4. So I think I should use sampler=DistributedSampler(dataset) and pass a batch_size to DataLoader (or use sampler=BatchSampler(DistributedSampler(dataset))). But how can I make it random? shuffle is mutually exclusive from sampler and I cannot mix RandomSampler with DistributedSampler.
2

Looking at the code DistributedSampler(RandomSampler(...)) is valid, dataset does not have to be a Dataset, it can be another Sampler because only len() matters that any Sampler provides.

3

@fmassa I am reading here you have added a new DistributedSampler that also has shuffle. Why not just pass sampler=DistributedSampler(RandomSampler(dataset)) to a DataLoader?

I want to see if there is any particular issue with above.