Guidelines for assigning num_workers to DataLoader

69

I implemented the algorithm to find optimal num_workers for fast training.

You can simply find optimal num_workers on any system with this algorithm.

The below code is example code.

import torch
import nws

batch_size = ...
dataset = ...

num_workers = nws.search(dataset=dataset,
                                 batch_size=batch_size,
                                 ...)

loader = torch.utils.data.DataLoader(dataset=dataset,
                                     batch_size=batch_size, 
                                     ...,
                                     num_workers=num_workers, 
                                     ...)