If you use num_workers > 1 you are already creating multiple workers, where each worker will load a batch and push it to the queue.
If you are still seeing a slowdown in loading the data, it would mean that your current data loading pipeline is still not fast enough.
This post gives some insights why data loading can be the bottleneck and some best practices.
1 Like