If my dataset has 28,000 images each of them 224 by 224 pixels, (results in around 350 MB of data)
and my GPU has 12 GB memory. Will PyTorch DataLoader load all the data in the-memory and keep it there?
I am trying to understand whether it is possible to pass the whole dataset in-memory to another application. Or does PyTorch keeps only part of the data in memory at all times?
I would like to understand how dataset size and GPU or CPU memory size (and possible batch size) can influence how much data is in memory?
Thank you
The DataLoader doesn’t define, if the data is loaded to the device or the CPU, which is specified in the Dataset.
Depending on your Dataset, you could either pre-load the dataset onto the GPU (in its __init__) or lazily load the data in the __getitem__ and push each batch to the GPU inside the training loop.
Also note that you might have an error in the size calculation.
Assuming you are dealing with 28.000 images in the spatial resolution of 224x224, the size would be:
# grayscale stored as 32bit floats:
28000 * 224 * 224 * 4 / 1024**3
> 5.23 GB
# RGB images stores as 32bit floats:
28000 * 3 * 224 * 224 * 4 / 1024**3
> 15.70 GB
Given this size, I would recommend to lazily load the data and push each batch to the device as described in the data loading tutorial.