I have a current setup where I create a few data loader with different batch sizes and number of workers which I use for different purposes. I have one for the normal training, and then another one for visualization, etc; I need different batch sizes.
Question: I was wondering if a dataloader, e.g. created with 8 workers, block memory, although they are not currently used? Or is it fine to generate a few data loaders, although some are only rarely used?
If you “execute” the DataLoader, i.e. start an iteration, the workers will be spawned, which will create copies of the underlying Dataset and will start creating the batches and adding them to the queue.
The batches will use memory and in case you are loading data in the Dataset.__init__ method, these object would also allocate memory.
If you are creating the DataLoader object without iterating it, the memory usage should be minimal.