Help with building faster images loader

1

Hi, just for education purposes I downloaded MNIST dataset in zip. version as jpg. files. And I made my own images loader for these images. I am loading these images from folders with PIL library, then covert them to Numpy and then convert them to a torch.tensor. Of shape (64,1,28,28)

Then I created normal python list and this list have these tensors in it.
(very long array of torch.tensors of shape - (64,1,28,28))

Then iam looping inside this array and iam pushing these tensors to model.

I want to ask you if you have any idea where can I store this tensors if I don’t want to store them in that array ? Any better way ? Or any tips how to speed up this images loader ?

2

Using Datasets and DataLoaders is the way to do this in PyTorch. An MNIST DataSet is part of torchvision.