About multiprocessing in dataloader when num_workers > 0

data
1

Hello, I use Pytorch for my research to train the machine-learning model.

As I know, “Dataloader” in PyTorch makes subprocesses and parallelizes the job that loads the data from drives to memory when the attribute “num_worker” is larger than zero.
However, I am a little bit confused that I understand exactly the process.
Are all subprocesses involved in loading data every time, or does each subprocess load and prepare data sequentially?

Best.

2

The main process generates the indices using Sampler and sends them to the workers, which then prepare the specific sample and send the result back to the main process.