CrossEntropyLoss gives error with conv1d but not with linear

1

I use CrossEntropyLoss where my labels have 2 classes and input is 1x38. When I use a linear model, everything runs fine. But when I use conv1d, I get an error.
Here is the linear info:

class ANN_shallow(nn.Module):
    def __init__(self):
        super(ANN_shallow,self).__init__()
        self.fc1 = nn.Sequential(nn.Linear(1*38, 1000), nn.ReLU())
        self.fc2 = nn.Sequential(nn.Linear(1000, 2), nn.Softmax(dim=1))
        
    def forward(self, x):
        print('input shape: ', x.shape)
        x = self.fc1(x)
        x = self.fc2(x)
        return x

input shape:  torch.Size([512, 38])
label shape:  torch.Size([512])
output shape:  torch.Size([512, 2])

So the linear model above runs without complain (not with impressive accuracy though :unamused:)

Here is the conv1d info:

class CNN1d_shallow(nn.Module):
    def __init__(self):
        super(CNN1d_shallow,self).__init__()
        self.conv1  = nn.Sequential(nn.Conv1d(1, 5, 3), nn.ReLU())
        self.fc1    = nn.Sequential(nn.Linear(36, 2), nn.Softmax(dim=1))

   def forward(self, x):
        x = x.unsqueeze(1)
        print('input shape: ', x.shape)
        x = self.conv1(x)
        x = self.fc1(x)
        return x

# printouts:
input shape:  torch.Size([512, 1, 38])
label shape:  torch.Size([512])
output shape:  torch.Size([512, 5, 2])

# error:
Traceback (most recent call last):
  File "D:\Users\wardah_w\Desktop\SimpleANN\src\experiment.py", line 115, in <module>
    loss            = criterion(outputs, labels)
  File "D:\Users\wardah_w\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "D:\Users\wardah_w\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\loss.py", line 904, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "D:\Users\wardah_w\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\functional.py", line 1970, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "D:\Users\wardah_w\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\functional.py", line 1800, in nll_loss
    out_size, target.size()))
ValueError: Expected target size (512, 2), got torch.Size([512])

I can see it’s the loss function that’s giving the error, and I can see differences in the output shapes. But the error says Expected target size (512, 2), got torch.Size([512]). I don’t get it. Can you please help me work this out? Thanks. :pensive:

2

I found the solution (Thank you DeepLizard)
I needed to flatten my conv1 output properly before passing into the fc1 layer.
:slight_smile:

3

Good to hear you’ve figured it out!
However, there seems to be another small issue in your code.
nn.CrossEntropyLoss expects raw logits as the model output, since internally F.log_softmax will be applied on them. Currently both models use nn.Softmax at their output, which might slow down the training significantly or just completely stop it.
Could you just remove the nn.Softmax and run your models again?