After reading the some codes of torch.utils.data.dataloader, I find it does not work like caffe, which prefetches next batch data during the GPUs are working. I find a blog which tries to do it. I will try it.
######################################################
After I read the blog, I made a mistake which the dataloader tries to prefetch the next batch data . But I find that it can NOT make full use of CPUs (it shows that it only uses about 60% ability of CPU). In the blog it shows that the data precoessing takes less than 18% of time. Actually, if it could fully achieve its target and made full use of CPUs and the disk is fast enough, it should be near 0%, not 18%.