Is there any code to load ImageNet 64x64, or 32x32 in PyTorch?
Just use the standard ImageFolder dataset class, and add a transform that resizes the image to 64x64 or 32x32? This will be a bit slower than having converted straight away the images to 32x32 and loading them, but it’s much more flexible (and saves space in disk!)
http://www.image-net.org/ provides the 64x64 and the 32x32 versions of ImageNet, as described by Chrabaszcz et al, “A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets”.
After downloading, the training set has 10 train_data_batches (train_data_batch1,…,10), one batch for the validation data, and the size of all the data is 14.2 GB.
So, I think there is someway to load them, without downloading the original ImageNet data, and then, resizing it. No?
Did you find a way to load the down-sampled dataset? I am trying the same by trouble getting it done.
I just downloaded the 64x64 version, and I’m thinking about loading each batch into GPU memory (one at a time), to eliminate any potential bottleneck of a data transfer on each forward pass. What you think - is this worth doing? Each batch is 1.3GB so I’d be wasting ~10% of memory.