Num_workers effect

1

I know that setting num_workers > 0 in Dataloader may speed training and data access. According to documentation num_workers is about

how many subprocesses to use for data loading. 0 means that the data will be loaded in the main process.

Does the loading refer to loading data into the RAM? For instance, suppose that there is enough memory to read the dataset; hence, our data is read from a text file when creating an instance of Dataset class. Now if we use num_workers > 0 when defining Dataloader, do we gain any advantage of these workers?

2

If your data is already loaded, you might get an advantage of using multiple workers for the preprocessing of each sample. If you don’t need to process the samples, but would simply slice the data, you won’t see a huge performance benefit. Just in case, you could nevertheless profile the code and verify it.