Does pytorch have some run time data generator ? like data generator in keras?

Is is possible to generate batch data at the running time using pytorch?
for example , my training data consists of 30000 256*256 images, can I generate batch data containing, for example ,10 images at each iteration, thus I don’t have to load the whole 30000 images one time but load them step by step?
Thank you.


have you looked at our examples much? most of the vision ones do this:


Hi ,
Did you find what you were looking for ?

The Dataset & DataLoader tutorial explains how lazy loading can be used and how the DataLoader creates the batches, so you might want to take a look at it. :slight_smile:

1 Like