ValueError: Expected target size (32, 7), got torch.Size([32])

#1

Hi everyone,
I know this topic was previously discussed, however, the proposed solutions didn’t work for me.
I am trying to perform classification of precomputed features into 7 categories using logistic regression.
I got the following error when training the classifier:

ValueError: Expected target size (32, 7), got torch.Size([32])

My target shape was ([768,1]) and squeezing it didn’t solve the problem.
The input shape also is torch.Size([768, 1, 221])

By squeezing it, I got this error:

RuntimeError: Expected object of scalar type Long but got scalar type Int for argument #2 'target'

To train the logistic regression model, I used this piece of code which works steadily with another dataset:

#define classifier
num_input = trainingData.shape[-1]
num_classes = trainingLabels.cpu().unique().numel()
model = Sequential(Linear(num_input, num_classes), LogSoftmax(dim=1))
optimizer = Adam(model.parameters())
criterion = NLLLoss()

batch_size = 32
num_epochs = 50
#learning rate
lr = 1e-4

nsamples = trainingData.shape[0]
nbatches = nsamples // batch_size

for e in range(num_epochs):
    perm = torch.randperm(nsamples)

    
    for i in range(nbatches):
        idx = perm[i * batch_size : (i+1) * batch_size]
        model.zero_grad()
        resp = model.forward(trainingData[idx])
        trainingLabels = trainingLabels.squeeze()
        loss = criterion(resp, trainingLabels[idx])
        loss.backward()
        optimizer.step()
    
    resp = model.forward(trainingData)
    avg_loss = criterion(resp, trainingLabels)

Obviously, my problem is in the data shape but I can not fix it may be because I am new to pytorch.

Any help will be appreciated.

#2

Hi Salma,

Based on the error message, it seems you have an additional dimension in your model output.
If you just would like to classify the data in 7 classes (not-pixel wise classification etc.), your output should have the shape [batch_size, nb_classes], while your target should be a torch.LongTensor containing the class indices in the shape [batch_size].

Try to squeeze the model output and convert your target as:

target = target.long()

If that doesn’t help, could you post the input, output and target shapes, so that we could debug this issue further?

Also a small issue in your code:
You should call the model directly (output = model(data)), since this will make sure to register all hooks etc. Currently you are using model.forward(data) which might yield some issues in the future.

While your code should work fine using your manual batching approach, you could use a Dataset and DataLoader instead, which will make shuffling, preprocessing the data etc. a bit easier.

2 Likes
CrossEntropyLoss Expected Target Size error
#3

Yes there was an additional dimension in the input and the target.
I squeezed the input and I converted the target to target.long

And it worked. Thank you!

#4

I have a similar error I am posting it here because it is similar and
I tried this method and its not working.
I was trying to do cross entropy loss and I got this error “Expected target size (1, 5), got torch.Size([1, 17451])”
My cls_preds.size()= torch.Size([1, 17451, 5]) and cls_targets.size()= torch.Size([1, 17451])
and I was trying to calculate Loss by following code
loss = nn.CrossEntropyLoss()
output = loss(cls_preds, cls_targets)

Please provide some explanation, thank you.

#5

nn.CrossEntrolyLoss expects a model output in the shape [batch_size, nb_classes, *additional_dims] and a target in the shape [batch_size, *additional_dims] containing the class indices in the range [0, nb_classes-1].

Based on your output shape it seems you are dealing with 17451 classes and a temporal dimension of 5. The target should thus have a shape of [1, 5] (note the missing channel dimension).

Also, it seems your target might be one-hot encoded, as it’s using the class dimension in dim1, but is missing the temp. dim?

Maybe your use case is vice versa and you are dealing with 5 classes.
In this case, you would have to permute the output so that it has the shape [1, 5, 17451].

1 Like
#6

Yes you are right my class_num = 5 and number of batches = 1 and the other dimension is number of anchors.
and I did cls_preds.permute(0,2,1) and my shape is now [1, 5, 17451].

Now I am getting an error saying “RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes’ failed. at /pytorch/aten/src/THNN/generic/SpatialClassNLLCriterion.c:111” in nn.nll_loss2d.

any help?
Thank you.

#7

If you are dealing with the shape [1, 5, 17451], I assume that each “anchor” belongs to one of the 5 classes?
If so, then the target should contain class indices in the range [0, 4].

1 Like
#8

Actually I tried number of anchor boxes as batch number and it got solved. It make sense? Thanks anyway.

#9

Hi, I face the same problem as Naveen, shape of my output is torch.Size([1, 20, 4]), where 20 is the time sequence, 4 is the number of classes, softmax was used as the final activitation function, and loss is nn.CrossEntropyLoss(), I use the follow expression:
loss = loss_func(output, torch.max(label, 2)[1])
I got this error:
ValueError: Expected target size (2, 4), got torch.Size([2, 20])
But if I changed my time sequence to 4, this error disappeared, why？

#10

That would be wrong as explained here:

so you would have to permute the model output such that the class dimension is in dim1.

That is unfortunately also wrong, since nn.CrossEntropyLoss expects logits, so you would have to remove the softmax.

#11

You are right, I added transpose op before output and it works. Thanks a lot.