ImageFolder with Custom Data

vision
1

I’m working with a custom data set in the format

folder
│     
│
└--train
    └--──class1
    |         │   file011
    |         │   file012
    |
    |
    └───--class2
          │   file021
          │   file022

└--val
    └--──class1
    |         │   file011
    |         │   file012
    |
    |
    └───--class2
          │   file021
          │   file022

When trying to load the dataset

data_dir = r'PATH_TO_DATA/train'

dataset = datasets.ImageFolder(data_dir, ...)

FileNotFoundError: Found no valid file for the classes Cat, Deer, Dog, Human. Supported extensions are: .jpg, .jpeg, .png, .ppm, .bmp, .pgm, .tif, .tiff, .webp

The only issue I found similar to this was here, however in their case there seemed to be a .ipynb_checkpoints file which was causing the issue. It doesn’t appear to be the case here.

I also checked for hidden files, and made sure the extensions are acceptable.

Edit: An important piece of information that I didn’t realize was the issue seems to be the issue. I am hosting this data on a remote using Rclone, mounting my onedrive to access the data. When accessing the data directly, the dataset is read just fine. It seems to be an issues of ImageFolder accessing data via the remote access more than anything else.

2

Which data type are you using?
Based on the error message no valid file extension can be used to read the files, so maybe you would need to use a custom loader in the ImageFolder dataset in case your extension differs.

3

The dtype is torch.FloatTensor, I think I should remove this discussion because when using google drive to access the data, it works fine through Colab. Only when hosting the data through RClone, there seems to be some data corruption.

4

If you’ve stored tensors directly, the file extension should still fail as these types are supported:

Supported extensions are: .jpg, .jpeg, .png, .ppm, .bmp, .pgm, .tif, .tiff, .webp

which are all image formats.