Num_worker and prefetch_factor in DataLoader not scaling as expected

See this Queue in DataLoader does not behave as expected when using num_workers? - #5 by lkc1)