@michaelklachko - I would say the more is the better as long as you are not starving your PyTorch from getting CPU time. In your case, you have 3 CPU logical cores per GPU (if I understand your setup correctly). Then if you set more than 3 workers they would just fight for CPU time, I would suggest to try 2 and keep 1 free for the PyTorch itself. You can check how 3 and 1 would work.