Potential bug in ImageFolder?

vision
1

I’m using the ImageFolder class provided by Pytorch to load a dataset. The dataset is under a the folder

/home/user/faces_emore/imgs/

which contains all the subfolders to the images.

While training, my program exited with an unusual error:

FileNotFoundError: [Errno 2] No such file or directory: '/home/user/data/faces_emore/imgs/1:85/85337.jpg'

Which is completely out of the blue because this occurred in the second epoch - after the first epoch was completely with no errors. I searched for ‘85337.jpg’ in my dataset and found that it is under the subfolder

'/home/user/data/faces_emore/imgs/1285/85337.jpg'

meaning that the ‘2’ was somehow replaced with a ‘:’ during training…?
I’m curious if anyone has encountered this or anything similar.

Thanks

2

Are you able to reproduce it or does it occur sporadically?

3

Hi, I just found this bug, so I haven’t been able to reproduce it. I’ll try training without changing anything and see what happens. Only problem is that it takes around 6 hours to train a single epoch, so it’ll probably take some time.

4

If it’s a bug with ImageFolder, you do not need to run the training (or access the model). You could just run the ‘for’ loop over the dataset and spot the error.

1 Like
5

I’ve run a couple more runs and haven’t been able to reproduce it. I guess it must have been a random system error.