Hi,
Thank you very much for your answer!
Ok, I now understand why the workers fetch batches instead of samples.
Do the workers fetch batches for the next epoch before it starts, or the batches of an epoch only start being fetched when the epoch starts?
On another subject, I noticed that when I choose batch_size=64, worker 1 reads the first 64 indexes, worker 2 reads the next 64 indexes, and so on. Is there a way of having workers reading interleaved indexes?
For example, when num_workers = 3:
- worker 1 reads indexes 1, 4, 7, 10…
- worker 2 reads indexes 2, 5, 8, 11…
- worker 3 reads indexes 3, 6, 9, 12…
Thank you!