Criterion Target is out of bounds

#1

I found a model that looked appealing to me at:
https://towardsdatascience.com/how-to-train-an-image-classifier-in-pytorch-and-use-it-to-perform-basic-inference-on-single-images-99465a1e9bf5

As a start I copied nearly everything from this website.
Firstly I want to explain my train and test datasets.

My Train Dataset contains some chinese signs as bitmaps with unknown “meaning” ( I am looking for unicode escapes)
My Test Dataset contains the range of simplified chinese characters as bitmaps (escape sequences are known).

I thought it would be better to do it this way as there are more than 22000 signs in simplified chinese. For training I would have needed more than one image per sign resulting in like 130000 pictures for training.
I only have 225 unknown bitmaps. By doing so I only have 225*6 pictures for training which I thought would be way better for performance.

Here is where I changed the code from the website:

def load_train_test():

    normalize = transforms.Normalize(mean=[0.5,],
                                     std=[0.5,])

    img_transforms = transforms.Compose([transforms.Resize(224),
                                         transforms.CenterCrop(224),
                                         transforms.ToTensor(),
                                         normalize])

    train_data = datasets.ImageFolder(TRAIN_DS,
                                      transform = img_transforms)

    test_data = datasets.ImageFolder(TEST_DS,
                                     transform = img_transforms)

    train_loader = torch.utils.data.DataLoader(train_data, batch_size=64)
    test_loader = torch.utils.data.DataLoader(test_data, batch_size=64)

    return train_loader, test_loader

In the example he created labels from SubsetRandomSampler().

Now I receive following error message:

Exception has occurred: IndexError
Target 10 is out of bounds.

The error occurs when I try to calculate loss at

loss = criterion(logps, labels)

Here is the code snippet (line 7)

for epoch in range(epochs):
    for inputs, labels in train_loader:
        steps += 1
        inputs, labels = inputs.to(device), labels.to(device)
        optimizer.zero_grad()
        logps = model.forward(inputs)
        loss = criterion(logps, labels)
        loss.backward()
        optimizer.step()
        running_loss += loss.item()

        if steps % print_every == 0:
            test_loss = 0
            accuracy = 0
            model.eval()
            with torch.no_grad():
                for inputs, labels in test_loader:
                    inputs, labels = inputs.to(device), labels.to(device)
                    logps = model.forward(inputs)
                    batch_loss = criterion(logps, labels)
                    test_loss += batch_loss.item()

                    ps = torch.exp(logps)
                    top_p, top_class = ps.topk(1, dim=1)
                    equals = top_class == lavels.view(*top_class.shape)
                    accuracy = torch.mean(equals.type(torch.FloatTensor)).item()
        
            train_losses.append(running_loss/len(train_loader))
            test_losses.append(test_loss/len(test_loader))
            print(f"Epoch {epoch+1}/{epochs}.."
                  f"Train loss: {running_loss/print_every:.3f}.."
                  f"Test loss: {test_loss/len(test_loader):.3f}.."
                  f"Test accuracy: {accuracy/len(test_loader):.3f}")
            running_loss = 0
            model.train()
torch.save(model, 'chinese.pth')

In debug mode I can see that there are some labels generated. I really don’t know why this error occurs.
Maybe someone of you knows what I am doing wrong.
Thanks for your time!

#2

I assume you are using nn.CrossEntropyLoss as the criterion.
In that case, the model output should contain logits and have the shape [batch_size, nb_classes], while the target would have the shape [batch_size] and contain the class indices in the range [0, nb_classes-1].
The error is raised, if your target uses out-of-bounds class indices.
E.g. for 10 classes, the valid target values would be [0, 9], while you are trying to pass a 10 to the criterion.

Also, don’t call the forward method, as it will not call potentially registered hooks. Call the model directly instead: output = model(inputs).

#3

As criterion I use nn.NLLLoss.
How can I fix the worng number of classes?
I should have 225 classes so I’m not sure why its already failing at 10 classes.

Thank you for that hint, I changed the forward line

#4

I guess the output of your model is not containing logits for all 225 classes, which would be defined by the last layer (nn.Linear in case you are working on a multi-class classification).
Change the out_features to 225 and it should work.

#5

This is my model description. Should I change

model.fc = nn.Sequential(nn.Linear(2048, 512),
                         nn.ReLU(),
                         nn.Dropout(0.2),
                         nn.Linear(512, 10),
                         nn.LogSoftmax(dim=1))

to

model.fc = nn.Sequential(nn.Linear(2048, 512),
                         nn.ReLU(),
                         nn.Dropout(0.2),
                         nn.Linear(512, 225),
                         nn.LogSoftmax(dim=1))

Edit:
This creates the same error. Now Target 225 is out of bounds.
Could this happen because I do not specify targets while I’m loading the images?

#6

Okay I guess the part of

if steps % print_every == 0:

is the problem.
As my test dataset contains 22222 “classses”(22222 signs) I would need 22222 as out_feature. Since this has nothing to do with my problem I will delete it and figure out some evaluation process and then share my solution here again.
As I simply copied this from the website I didn’t think of its purpose. I guess the best solution to this problem would be if I generate some more images of those 225 unknown images and test the accuracy of this neural net with them so the labels are fitting.

#7

Your target index of 225 implies 226 classes (class indices would be in the range [0, 225], which are 226 values), so your model should also return 226 output logits.