Why is my dataset class giving index out of range errors

Hi, I am determining why my data set gives IndexError: list index out of range
error.

Consider this torch data set:

class MyDataset(Dataset):

def __init__(self, imgs , transform = None):
    self.imgs = imgs
    self.transform = transform or transforms.ToTensor()
    self.class_to_idx = {}

def __getitem__(self, index):
    
    image_path = self.imgs[index]
    name = image_path.split('/D')[1]
    target = name.split('_')[0]
    
    image = Image.open(image_path)
    
    if self.transform is not None:
        image = self.transform(image)

    if target in class_to_idx : 
        target = [class_to_idx[target]]
    else : 
        class_to_idx[target] = (int(target)-1)
        target = [class_to_idx[target]]


    return image , target

def __len__(self):
    return len(self.imgs)

An image_path example: "Train/D01_I_flat_0009.jpg"
The dataset has 35 classes: D01…D35

I want to calculate mean and std, but I get the error:
enter image description here
enter image description here

When I test the dataset, it works correctly:

enter image description here

Your image_path doesn’t seem to contain the expected /D string and thus the split call fails:

# works
image_path = "aaa/Dbbb"
name = image_path.split('/D')[1]
print(name)
# bbb

# fails
image_path = "aaabbb"
name = image_path.split('/D')[1]
# IndexError: list index out of range
1 Like

Dear @ptrblck, thanks for your reply.

All images’ names start with Dxx, and the full path of training images is ./Train/Dxx_***.jpg.
for example ./Train/D24_nature.jpg

As I mentioned, when I pass a batch (size=16) to the data loader, it correctly returns the labels and displays the images:

I found the reason for the error. It was an odd cause!
There weren’t any other files but .jpg files in the Train folder. However, I applied the following change:

train_files = glob.glob(train_dir + '/*')

to

train_files = glob.glob(train_dir + '/*.jpg')

The problem was solved!

It is worth mentioning that I didn’t face this problem on Google Colab, but with Jupyter Notebook on my machine.