Restarting a DataLoader before it ends?


(John) #1

I have a DataLoader instance that wraps a rather large dataset, but in some situations I just want to
get the first k batches from it, but I want to get those first k batches several times in a row (e.g. when doing validation after each epoch).

What is the correct way to do this especially if the dataloader uses multiprocessing workers?