ValueError: Expected input batch_size (150) to match target batch_size (50)

iter = 0
for epoch in range(n_epoch):
for i, (images, labels) in enumerate(train_loader):

    #######################
    #  USE GPU FOR MODEL  #
    #######################
    if torch.cuda.is_available():
        images = Variable(images.view(-1, 28*28).cuda())
        labels = Variable(labels.cuda())
    else:
        images = Variable(images.view(-1, 28*28))
        labels = Variable(labels)
    
    # Clear gradients w.r.t. parameters
    optimizer.zero_grad()
    print(images.shape)
    # Forward pass to get output/logits
    outputs = model(images)
    print(outputs.shape)
    # Calculate Loss: softmax --> cross entropy loss
    loss = criterion(outputs, labels)
    
    # Getting gradients w.r.t. parameters
    loss.backward()
    
    # Updating parameters
    optimizer.step()
    
    iter += 1
    
    if iter % 500 == 0:
        # Calculate Accuracy         
        correct = 0
        total = 0
        # Iterate through test dataset
        for images, labels in test_loader:
            #######################
            #  USE GPU FOR MODEL  #
            #######################
            images = Variable(images.view(-1, 12288).cuda())
            
            # Forward pass only to get logits/output
            outputs = model(images)
            
            # Get predictions from the maximum value
            _, predicted = torch.max(outputs.data, 1)
            
            # Total number of labels
            total += labels.size(0)
            
            #######################
            #  USE GPU FOR MODEL  #
            #######################
            # Total correct predictions
            correct += (predicted.cpu() == labels.cpu()).sum()
        
        accuracy = 100 * correct / total
        
        # Print Loss
        print('Iteration: {}. Loss: {}. Accuracy: {}'.format(iter, loss.data, accuracy))

I’m new in pytorch. My model is initialized as such

input_dim = 28*28
output_dim = 2
model = FeedForward_NN(input_dim,output_dim)

I get a ValueError: Expected input batch_size (150) to match target batch_size (50)
whats the reason for this?

Could you post the definition of your model so that we can have a look, please? :slight_smile:

class FeedForward_NN(nn.Module):
def init(self,input_dim,output_dim):
super(FeedForward_NN,self).init()

    self.fc0 = nn.Linear(input_dim, output_dim)
    
def forward(self,x):

    out = self.fc0(x)
    return out

Thanks for the code.
It looks like this line of code is creating an invalid view:

images = Variable(images.view(-1, 12288).cuda())

Based on the model definition and the training loop, each input should have a shape of [batch_size, 28*28=784].

Also, Variables are deprecated since PyTorch 0.4.0 so you can simply use tensors now. :wink:

ahh. Sorry, I changed the code while trying to figure out the error. The error exists nonetheless. Even after changing it to

images = Variable(images.view(-1, 28*28).cuda())

Could you print the shapes of your input as well as the target before passing them to the model, please?

So these are my parameters shape

print(model.parameters())

print(len(list(model.parameters())))

print(list(model.parameters())[0].size())

print(list(model.parameters())[1].size())

output:

<generator object Module.parameters at 0x7fc5eb5a6fc0>
2
torch.Size([2, 784])
torch.Size([2])

The shape it takes after using DataLoader:

for i, (images, labels) in enumerate(train_loader):
print(images.shape)

output:
This goes on for 36 times

torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])
torch.Size([50, 3, 28, 28])

images = Variable(images.view(-1, 28*28).cuda())
image size after the above line
output:
torch.Size([150, 784])

outputs = model(images)
outputs size after the above line
output:
torch.Size([150, 2])

Thanks for the information.
The input shapes don’t match your view call inside the model, as you are pushing the 3 channels to the batch dimension.
Could you use images.view(images.shape(0), -1) to create the views?
This will create an input tensor of [batch_size, 28*28*3 = 2352], so that you would also need to adapt the input features of your linear layer.

Also, based on your initial description it seems you expect to deal with single-channel images, so are you transforming them to RGB somewhere?

so the images where originally in torch.Size([28, 28, 3]). But as pytorch takes input in [channels, height, width]. I changed it to torch.Size([3, 28, 28]) using the permute() function.

And using images.view(images.shape(0), -1) gives the following error

TypeError Traceback (most recent call last)
in ()
7 #######################
8 if torch.cuda.is_available():
9 images = Variable(images.view(images.shape(0),-1).cuda())
10 labels = Variable(labels.cuda())
11 else:

TypeError: ‘torch.Size’ object is not callable

Oh, my bad. Use images.size(0) instead of .shape.

I get a cuda error if I do that :confused:


RuntimeError Traceback (most recent call last)
in
23
24 # Getting gradients w.r.t. parameters
—> 25 loss.backward()
26
27 # Updating parameters

~/anaconda3/lib/python3.7/site-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)
164 products. Defaults to False.
165 “”"
→ 166 torch.autograd.backward(self, gradient, retain_graph, create_graph)
167
168 def register_hook(self, hook):

~/anaconda3/lib/python3.7/site-packages/torch/autograd/init.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
97 Variable._execution_engine.run_backward(
98 tensors, grad_tensors, retain_graph, create_graph,
—> 99 allow_unreachable=True) # allow_unreachable flag
100
101

RuntimeError: CUDA error: device-side assert triggered

Also I’m giving my input dimensions as such:

input_dim = 28*28*3
output_dim = 2
model = FeedForward_NN(input_dim,output_dim)

Could you run the code on the CPU and check the error message, please?
This should give you a better error.
If the code runs fine on the CPU, use CUDA_LAUNCH_BLOCKING=1 python setup.py args to rerun the script on the GPU again and post the stack trace here.

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-21-874ab4eae012> in <module>
     20         print(outputs.shape)
     21         # Calculate Loss: softmax --> cross entropy loss
---> 22         loss = criterion(outputs, labels)
     23 
     24         # Getting gradients w.r.t. parameters

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--> 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/loss.py in forward(self, input, target)
    914     def forward(self, input, target):
    915         return F.cross_entropy(input, target, weight=self.weight,
--> 916                                ignore_index=self.ignore_index, reduction=self.reduction)
    917 
    918 

~/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
   2007     if size_average is not None or reduce is not None:
   2008         reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 2009     return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
   2010 
   2011 

~/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
   1836                          .format(input.size(0), target.size(0)))
   1837     if dim == 2:
-> 1838         ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
   1839     elif dim == 4:
   1840         ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes' failed.  at /opt/conda/conda-bld/pytorch_1573049306803/work/aten/src/THNN/generic/ClassNLLCriterion.c:97

I get the above error on cpu

Thanks for the error message.
It seems your targets are out of bounds.
For 2 output units, your target should be a LongTensor with the shape [batch_size] containing class indices in [0, 1].

This is how my labels look

tensor([1, 2, 2, 1, 1, 2, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 1, 1, 2, 1, 2, 2, 2,
        2, 1, 1, 2, 2, 2, 2, 1, 2, 1, 1, 1, 1, 2, 1, 2, 1, 1, 2, 2, 2, 1, 1, 2,
        2, 2])

And the shapes are

labels
torch.Size([50])
images
torch.Size([50, 2352])
outputs
torch.Size([50, 2])

Seems to have the same size as the batch_size

The shape seems to be correct, but the labels should be zeros and ones.

Yes It worked!! Thank you so much for all the help