Handling large 3d image dataset with DataLoader


I am working with MSD (Medical Segmentation Decathlon) dataset for a segmentation problem.
In Dataset’s init method I am storing all filenames in list, and in getitem reading the files, applying transformation, etc…

I am facing some issue with GPU utilization:
Problem is, nvidia-sme command is showing only 1 process (whereas I have specified 4 workers in dataloader) running, with GPU memory usage as 4GB. Secondly there is no GPU utilization at all.

I was reading an article which says that if DataLoader is slow, model will run on CPU. But how can I speed up the dataloader and run everything on GPU

The num_workers specified in the DataLoader are executed on the CPU using multiprocessing and will not be shown on nvidia-smi.
The GPU utilization might be low, if you are facing bottlenecks in your code, such as data loading.
Take a look at this post, which explains it well and suggests some performance improvements.

This is not correct. The model will use the specified device and PyTorch will not automatically use the CPU based on the performance of the DataLoader.

1 Like