drop_last=False and the last batch of data cannot be evenly distributed to each GPU

distributed
1

In the multi-GPU dataloader, if I set drop_last=False and the last batch of data cannot be evenly distributed to each GPU, what will PyTorch do?

2

The DistributedSampler will drop the tail of the samples as seen here.

1 Like
3

I understand, thank you!