That’s not the case since each worker will start loading the next batch once the number of batches waiting in the queue is not meeting the prefetch factor.
This code snippet illustrates it.
As you can see the creation of the iterator will directly load prefetch_factor * num_workers = 2 * 8 = 16 batches, so nb_batches * batch_size = 16 * 2 = 32 samples.
The next call then returns a batch and you can see that one of the workers directly starts to load the next batch (2 samples) to fill the queue.
2 Likes