I tried to google the answer, but couldn’t find it.
i) Is each worker in the the dataloader calling __ getitem __ within the same batch, and then collating it,
ii) or is each worker loading and collating each batch independently?
Imagine this scenario:
16 batches per epoch, each with 100 examples.
24 workers available.
Will 8 workers be iddle if I set num_workers = 24?