Questions about Dataloader and Dataset

1

I run my dataloader like this:

dataset= ImageFolder('/home/x/data/pre/train/',
                      transform=transforms.Compose([transforms.Scale(opt.image_size),
                                              transforms.RandomCrop(opt.image_size) ,
                                              transforms.ToTensor(),
                                              transforms.Normalize([0.5]*3,[0.5]*3)
                                             ]))
 
dataloader=t.utils.data.DataLoader(dataset,opt.batch_size,True,num_workers=opt.workers)

but some images are corrupted, and it raised an error(an PIL error in default_loader: Image.open(path).convert('RGB')

my temporary fix is modifying /torchvision/datasets/folder.py Line 65 to

        try:
            img = self.loader(os.path.join(self.root, path))
        except Exception as e:
            index = index - 1 if index > 0 else index + 1 
            return self.__getitem__(index)

Is there a better way to solve this, i.e. modifying the code of dataloader to load another Image when exception or write a new loader.
or does it work if I simply return None when caught an exception?

11 Likes