Details on how workers prefetch data?

data
1

Hi, lets assume that num-workers = 8 and batch_size=2.
So, 2*8 = 16 number of batches have to be pre loaded .

Now, when the current batch is consumed by a worker and next is called on the data loader, how many new batches does it fetch? As I understand, 1 new batch is fetched for every consumed batch to maintain the constant count of 16 batches each time. Is this correct ?

Do the number of batches always remain 16 and for every batch consumed, a new batch would be fetched ?

1 Like
2

This would be the case if the default prefetch_factor of 2 is used but note that the 2 is not coming from the batch_size.
You can take a look at this code snippet to see how the prefetch_factor works.

1 Like