Have a look at the Data Loading tutorial.
Basically your data can be lazily loaded in Dataset
's __getitem__
method, if you for example provide the image paths.
In __getitem__
you should return two tensors, i.e. the data and the corresponding target.
By wrapping the Dataset
into a DataLoader
multiple worker can be used to load and preprocess the data.
Your for loop will automatically work, if you use for input, target in loader:
.
Here is a better explanation.