Is is possible to generate batch data at the running time using pytorch?
for example , my training data consists of 30000 256*256 images, can I generate batch data containing, for example ,10 images at each iteration, thus I don’t have to load the whole 30000 images one time but load them step by step?
Thank you.
2 Likes
have you looked at our examples much? most of the vision ones do this: https://github.com/pytorch/examples/
2 Likes
Hi ,
Did you find what you were looking for ?
The Dataset & DataLoader tutorial explains how lazy loading can be used and how the DataLoader creates the batches, so you might want to take a look at it. 
1 Like