Imagenet folders problem

vision
1

The imagenet dataset that I downloaded from ILSVRC has two sub-folders

  1. Annotations
  2. Data
  3. ImageSets
    Out of the above 3, the second folder, i.e. “Data” folder seems to have the .jpeg files.
    This folder has a single subfolder called CLS-LOC, which further has 3 subfolders

2 a) test
2 b) train
2 c)val

While training and testing, I use the following for loading datasets

traindir = '/home/ec2-user/anaconda3/ILSVRC/Data/CLS-LOC/train'
trainset = datasets.ImageFolder(traindir,t_train)    
train_set = torch.utils.data.DataLoader(trainset,batch_size=bsz,shuffle=True,num_workers=4,pin_memory=True)

while for validation, I use the following

testdir = '/home/ec2-user/anaconda3/ILSVRC/Data/CLS-LOC/test'
testset = datasets.ImageFolder(valdir,t_test)
test_set = \
torch.utils.data.DataLoader(testset,batch_size=bsz_test,shuffle=False,num_workers=2,pin_memory=True)

However, I get the following error

“RuntimeError: Found 0 files in subfolders of: /home/ec2-user/anaconda3/ILSVRC/Data/CLS-LOC/test
Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif”

I am confused what to do?

2

Could you check, if the download was successful?
Are you seeing files with a size > 0kB?

3

The val folder size is 6.4 G, and train folder size is approximately 280 G (which includes tar files, and also their extracted version).

4

What does ls /home/ec2-user/anaconda3/ILSVRC/Data/CLS-LOC/test in your terminal return?

5

it gives me the following error

““RuntimeError: Found 0 files in subfolders of: /home/ec2-user/anaconda3/ILSVRC/Data/CLS-LOC/test
Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif””

6

This seems to be the error thrown by ImageFolder.
Could you type the command into your terminal (bash) and see, if there are folders inside the path?

7

I didnt understand…I am attaching the snapshot of my val folder here…please find it attached here…

Screenshot%20from%202018-06-26%2013-16-07
Screenshot%20from%202018-06-26%2013-16-071318×963 205 KB

8

What’s inside these folders?
Could you look into some with ls?

9

Hi, I am using the 2012 dataset now…there doesnt seem to be the same issue…i did an ls, and am attaching screenshot of results…however, the accuracy on train and val tests are differing a lot (50% versus 1%).

Screenshot%20from%202018-06-26%2015-14-27
Screenshot%20from%202018-06-26%2015-14-271330×300 70 KB

10

Maybe there are still some empty folders?
Could you try to find them with find . -type d -empty?

11

I typed the above command in the val folder, and hit enter…it printed nothing (I am assuming it means all folders are non empty)…also, a print out of the total count of the validation set images while validating, is around 50k…

12

Hi
I have faced same problem when I’m trying to load data which is present in a folder. The thing is create a folder again in the target folder with that data… Now in your case /test/test_dummy…
Test_dummy contains same as your test data. So that torch ImageFolder can find test_dummy as a subfolder inside the test folder. Now it doesn’t give any error. Even though larger datasets require more time to copy there is no other option I have found. I hope you find it useful.
Thank you. Have a try and let me know.