Target size (torch.Size([32])) must be the same as input size (torch.Size([32, 24]))

data_transforms = {
    'train': transforms.Compose([
    'valid': transforms.Compose([
class Dataset_load(Dataset):
    def __init__(self, file_path, transform=None): = pd.read_csv(file_path)
        self.transform = transform
    def __len__(self):
        return len(
    def __getitem__(self, index):
        image =[index, 1:].values.astype(np.uint8).reshape((28,28,1))
        label =[index, 0]
        if self.transform is not None:
            image = self.transform(image)
        return image , label


train_loader =, batch_size=32, shuffle=True)


test_loader, batch_size=32, shuffle=True)

import torch.nn.functional as F

class Net(nn.Module):
  def __init__(self):
    #input depth , output depth , kernel size(filter)x
    self.conv1=nn.Conv2d(1,16,kernel_size=(3, 3),padding=(1, 1),stride=(1, 1))
    self.conv2=nn.Conv2d(16,32,kernel_size=(3, 3),padding=(1, 1),stride=(1, 1))
    self.conv3=nn.Conv2d(32,64,kernel_size=(3, 3),padding=(1, 1),stride=(1, 1))
    #padding for last conv layer 
    self.adapt = nn.AdaptiveMaxPool2d((4,4))  
    #padding layer
    #dropout layer
    #fc layers 
  def forward(self,x):
    #flatten Images
    x = x.view(x.size(0), -1)
    return F.log_softmax(x)

model = Net()
optimizer = optim.Adam(model.parameters(), lr=0.001)
criterion = nn.BCEWithLogitsLoss()
exp_lr_scheduler = lr_scheduler.StepLR(optimizer, step_size=4, gamma=0.1)
if torch.cuda.is_available():
    model = model.cuda()
    criterion = criterion.cuda()

def train(n_epochs=100):

  for epoch in range(1, n_epochs+1):

    # keep track of training and validation loss
    train_loss = 0.0
    for data, target in train_loader:
        if torch.cuda.is_available():
            data, target = data.cuda(), target.cuda()
        # clear the gradients of all optimized variables
        # forward pass: compute predicted outputs by passing inputs to the model
        output = model(data)
        # calculate the batch loss
        loss = criterion(output, target)
        # backward pass: compute gradient of the loss with respect to model parameters
        # perform a single optimization step (parameter update)
        # update training loss
        train_loss += loss.item()*data.size(0)
    train_loss = train_loss/len(train_loader)
    # print training/validation statistics 
    print('Epoch: {} \tTraining Loss: {:.6f}}'.format(
        epoch, train_loss))
torch.Size([32, 1, 28, 28])
/usr/local/lib/python3.6/dist-packages/ UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
ValueError                                Traceback (most recent call last)
<ipython-input-73-c813bd3a9aca> in <module>()
      1 from torch.autograd import Variable
----> 2 train(10)

3 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/ in binary_cross_entropy_with_logits(input, target, weight, size_average, reduce, reduction, pos_weight)
   2160     if not (target.size() == input.size()):
-> 2161         raise ValueError("Target size ({}) must be the same as input size ({})".format(target.size(), input.size()))
   2163     return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)

ValueError: Target size (torch.Size([32])) must be the same as input size (torch.Size([32, 24]))

If you are dealing with a multi-class classification use case and your target tensor contains class indices, you should use nn.NLLLoss as your criterion.
Alternatively, you could also remove the F.Log_softmax from your model and use nn.CrossEntropyLoss (identical to F.log_softmax + nn.NLLLoss).

However, if you are dealing with a multi-label classification (each sample can have more than a single active class), you should remove the F.log_softmax and would have to provide a target for each class output.

1 Like

thanks for replying
I removed log_softmax and used CrossEntropyLoss but I got this Error

Assertion `cur_target >= 0 && cur_target < n_classes' failed.  at /pytorch/aten/src/THNN/generic/ClassNLLCriterion.c:92
1 Like

Your target should contain class indices in the range [0, nb_classes-1].
In your case this would be [0, 23].

To debug this issue, you could add a print statement and check, which target batch fails this assumption.

1 Like

Please could you tell about doing it on nn.BCEWithLogitsLoss()

def real_loss(D_out):
   labels = torch.ones(D_out.size(0)) * 0.9 #smoothing by default
    criterion = nn.BCEWithLogitsLoss()
    if train_on_gpu:
        labels = labels.cuda()
    loss = criterion(D_out.squeeze(), labels)
    return loss

It is giving error
ValueError: Target size (torch.Size([32])) must be the same as input size (torch.Size([32, 4096]))

But i tried to debug it, printed results and found that error were coming from using criterion(D_out.squeeze(), labels)

I printed out shape of D_out found that torch.Size([32, 4096])

I hit trialed any possible combination to accept criterion but it throws same error.

Please could you give me any idea why this kind of error is occuring in my code?
Thank you for helping us :slight_smile:

nn.BCEWihtLogitsLoss expects the model output and target to have the same shape.
Based on your provided shapes, it seems that your target might have the shape [batch_size] and might contain the class indices for each sample?
If that’s the case, you are dealing with a multi-class classification use case and should use nn.CrossEntropyLoss instead.

Based on your model output you might also be dealing with a multi-label classification use case, where each sample might belong to zero, one, or multiple classes. If that’s the case, your target should contain values in the range [0, 1] and have the shape [batch_size, nb_classes].

1 Like

Thanks explaining , after changing to torch.Size([32, 1]) error was i was returning 4096 in discriminator.


I’m working on multi-label classification with 8 labels and BCEWithLogitsLoss as a loss function. My target is in the range [0, 7].
I’m getting this error and I can’t figure out what I should change…

Any help is appreciated.

     56             #labels = labels.unsqueeze(1)
---> 57             loss = loss_fn(logits, labels)

~/anaconda3/envs/env/lib/python3.8/site-packages/torch/nn/modules/ in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

~/anaconda3/envs/env/lib/python3.8/site-packages/torch/nn/modules/ in forward(self, input, target)
    627     def forward(self, input: Tensor, target: Tensor) -> Tensor:
--> 628         return F.binary_cross_entropy_with_logits(input, target,
    629                                                   self.weight,
    630                                                   pos_weight=self.pos_weight,

~/anaconda3/envs/env/lib/python3.8/site-packages/torch/nn/ in binary_cross_entropy_with_logits(input, target, weight, size_average, reduce, reduction, pos_weight)
   2537     if not (target.size() == input.size()):
-> 2538         raise ValueError("Target size ({}) must be the same as input size ({})".format(target.size(), input.size()))
   2540     return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)

ValueError: Target size (torch.Size([16])) must be the same as input size (torch.Size([16, 8]))

This sounds rather like a multi-class classification (one active class per sample) and you could use nn.CrossEntropyLoss instead.
nn.BCEWithLogitsLoss is used for a multi-label classification (zero, one, or multiple active classes per sample) and expects the model output and target in the same shape.

It’s multi-label, with more than one class per sample.

How does the target value range of [0, 7] fit this use case, i.e. if the target contains an integer the the sample, how would you select e.g. two active classes?

Its text classification using BertModel, where one text can have multiple labels.
The idea of [0, 7] comes because I use PyTorch and the labels have to range from [0- n_labels-1].

Now I’m stuck because it’s said on Multilabel, nn.BCEWithLogitsLoss is applicable, but when I tried I get the above error…

That’s true for a multi-class classification use case using nn.CrossEntropyLoss or nn.NLLLoss, but wrong for a multi-label classification using nn.BCEWithLogitsLoss.

For the latter use case, your targets should have the same shape as the model outputs and be multi-hot encoded, so that you can set multiple classes to “active” or “inactive” (1 and 0).

Thanks for explaining,
I that case, what should I change to get the same shape?

Your targets would have to be already multi-hot encoded. If your current target shape is [batch_size] and the targets contain values in [0, nb_classes-1], then they are multi-class classification targets and you could use nn.CrossEntropyLoss as described before.

However, if you want to use these targets for a multi-label classification for specific reasons, you could use F.one_hot to create the expected shape for nn.BCEWithLogitsLoss.

Hi @ptrblck , How about I used criterion = nn.BCEWithLogitsLoss() in binary classification and get the Error: Target size (torch.Size([1, 2])) must be the same as input size (torch.Size([1, 1000]))

The posted shaped don’t match a binary classification case, as the input seems to have the shape [batch_size=1, nb_classes=1000], while the target has the shape [batch_size=1, nb_classes=2]. Based on the target shape it seems you are working on a multi-label classification. However based on the model output shape it seems you might have forgotten to replace the last linear layer of a pretrained classification model, so that the 1000 ImageNet class logits will be returned.

The error is raised, since nn.BCEWithLogitsLoss expects the model output and target so have the same shape.

1 Like