Yes, num_workers is the total number of processes used in data loading.
I’ve found here the general recommandation of using 4 workers per GPU, and I’ve found that it works really well with my own setup, but that might not be universal… @albanD’s method (adding more until it peaks) is probably the best way to find what works for you.