Questions related to Custom DataLoader

[Jh_R]

Thank you for your reply.

Well, there is no specific reason to set num_workers to 8, and I know that the more workers i know,
the faster I / O processes can handle them.
(I refer to the following article.)

Relation between num_workers, batch_size and epoch in DataLoader?

I have a question related to a relation between num_workers in DataLoader to batch_size and Epoch number. E.g. Lets assume we have total training size = 2000, and we use batch_size of 20 and we use 10 num_workers in DataLoader. Does this mean that DataLoader will return 20*10 = 200 examples at parallel which consist of 20 examples from each of the individual DataLoader’s workers ? Or DataLoader returns 20 examples sequentially from each worker and finish 10 epochs at parallel after 10(numb…

I checked as you told me, and I’ve confirmed that the number of num_workers+1 (main process + subprocess num_workers) pid is allocated. Is the usage wrong?