Shape of labels and shape of model(input) is different

1

Do I correctly understand that a shape of labels in classification and shape of output of model(input) is different?
From CIFAR-10 output = model(input), output:

tensor([[-0.3947, -1.8114,  0.5413,  1.2491, -0.0409,  2.1655, -0.7398,  0.8994,
         -0.7245, -0.6783],
        [ 5.3390,  3.6180, -0.4063, -2.5301, -2.6485, -3.5857, -3.7007, -3.5152,
          9.2215,  3.2827],
        [-1.0966, -3.1175,  1.1547,  2.9437,  0.3493,  3.4713,  0.2951,  1.2098,
         -1.7341, -2.9322],
        [-1.8294, -2.7313,  0.9235,  0.6192,  3.2165,  1.3419,  1.4856,  2.6439,
         -3.9445, -2.5060]], grad_fn=<ThAddmmBackward>)

Labels:

tensor([7, 3, 9, 8])

It’s contrintuitive to have.

2

It depends on your loss function.
If you use nn.CrossEntropyLoss or nn.NLLLoss for a classification use case, then you are right.
The output of your model should give logits (or log probabilities) for each sample in the batch and each class, i.e. its output will be [batch_size, nb_classes].
The targets however are just holding the class index for each sample in the batch, i.e. its shape will be [batch_size].

However, if you use another loss function, e.g. nn.MSELoss, the shapes of the model’s output and target should be the same.

The docs give an overview of the different loss functions and the expected shapes.

1 Like