What is pros of data.dataloader


(손주형) #1

DataLoader is a lot convenient to use.

But I’m wondering whether it is faster than custom python dataset loader.


(Bhushan Sonawane) #2

API overhead is not more than python dataset loader.
It depends on how you write the Custom Dataset.
If you load full data or buffer in init then getitem will be faster.
And if you want to keep RAM free, then load images in getitem


#3

Like @bhushans23 said, if you are just indexing pre-loaded data, you could probably use the Dataset directly. However, if you would like to lazily load or preprocess the data, DataLoader will provide multiprocessing to do so. Also, you can specify to use pinned memory so that the data transfer between the host and device will be faster.

Generally, I would stick to using DataLoader, as I’m not seeing any disadvantages.
Did you see any performance issues using a DataLoader over the plain Dataset?


(손주형) #4

no there was no issue.
I just wondered whether there is difference or not
.