CrossEntropyLoss: Index Error (Target 3 is out of bounds)

Hi there,

I am using code from a CIFAR classification problem (num_classes = 10) and want to use the code for my dataset (CheXpert with num_classes = 3). Therefore, I changed the num_classes in the ResNet model from 10 to 3.

class ResNet(FitModule):
    def __init__(self, block, num_blocks, num_classes=3):    #changed from 10 to 3
        super(ResNet, self).__init__()
        self.in_planes = 64
        self.conv1 = conv3x3(3, 64)
        self.bn1 = nn.BatchNorm2d(64)                                          
        self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1)      
        self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2)    
        self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2)    
        self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2)     

        self.ap = nn.AdaptiveAvgPool2d(1)
        self.mp = nn.AdaptiveMaxPool2d(1)

        self.linear = nn.Linear(1024 * block.expansion, num_classes)        
   ...
    def forward(self, x):
        x = x.float()                                               
        out = F.relu(self.bn1(self.conv1(x).float()).float())       
        out = self.layer1(out)                                      
        out = self.layer2(out)                                      
        out = self.layer3(out)                                      
        out = self.layer4(out)                                      
        out = torch.cat([self.mp(out), self.ap(out)], 1)           
        out = out.view(out.size(0), -1)                                   
        out = self.linear(out)
        return out

Then the IndexError: Target 3 is out of bounds occurs in my fit-methode when using CrossEntropyLoss.
10 pictures of size 3x32x32 are given into the model. That’s why X_batch has size [10, 3, 32, 32], after going through the model, y_batch_pred has size [10, 3] as I changed num_classes to 3.
When using the CrossEntropyLoss with y_batch_pred [10, 3] and the initial labels y_batch [10] the IndexError occurs.

    def fit(self, X, y, batch_size=32, epochs=10, verbose=1, validation_split=0.,             
              validation_data=None, shuffle=True, initial_epoch=0, seed=None,
              loss=CrossEntropyLoss(), optimizer=partial(SGD, lr=0.001, momentum=0.9),
              metrics=None):
        
        ...
            # Run batches
            for batch_i, batch_data in enumerate(train_data):
                # Get batch data
                X_batch = Variable(batch_data[0], requires_grad=True).float()
                y_batch = Variable(batch_data[1], requires_grad=True).long()
                # Backprop
                opt.zero_grad()
                y_batch_pred = self(X_batch).float()             # picture goes through model
                batch_loss = loss(y_batch_pred, y_batch)    # IndexError !!
                batch_loss.backward()
                opt.step()
                # Update status
                epoch_loss += batch_loss.item()
                for param in self.parameters():
                    param.requires_grad = True
                log['loss'] = float(epoch_loss) / (batch_i + 1)
                if verbose:
                    pb.bar(batch_i, log_to_message(log))
            ...
        return logs

There does not seem to be a big difference to between the original CIFAR problem with 10 classes and my problem with only 3 classes. When running my images on a 10 classes classification problem everything works just fine.

Can anyone help?

Thanks in advance,
Best,
Fabian

For 3 classes, your target should have the shape [batch_size] (which seems to be correct) and contain the values [0, 1, 2]. It seems you are using a value of 3, which will raise this error, as this would mean you are using 4 classes (note that the class index starts at 0).

Perfect, when importing my labels I changed them to 0,1,2 (before they were 1,2,3) and now it seems to work.
Thanks plenty!

Hi, That works for me too.

Thanks a lot.