- How does num_workers work under the hood?
- Does each worker load a new batch in parallel while other workers transfer their batch to the device? Or are all the workers waiting in line until all batches are transferred to the device, and only then will they start loading a new batch?
- Is this already implemented at the software level, or are there some homemade options for parallelizing the loading of batches?