Expected target size in loss

Hi everyone,

I’m used to other deep learning frameworks and it’s been a while since I’ve used PyTorch. Having a problem with my forward method. I viewed the other topics which were suggested but I couldn’t quite find anything helpful.

This is the error, 32 is the batch size in the dataloader:

return torch._C._nn.nll_loss_nd(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

RuntimeError: Expected target size [32, 3], got [32]

This is the model:

X = pad_sequence([torch.LongTensor(s) for s in df['Tweets']], batch_first=True, padding_value=0)
y = torch.tensor(df['Sentiments'].values)

trainCount = int(.7 * TotalObs)
validCount = int(.2*TotalObs)
testCount = TotalObs-trainCount-validCount
dataset = TensorDataset(X,y)
trainDataset, validDataset, testDataset = torch.utils.data.random_split(dataset, (trainCount, validCount, testCount))
train_dataset_loader = DataLoader(trainDataset, batch_size=BATCH_SIZE, shuffle=True)  
valid_dataset_loader = DataLoader(validDataset, batch_size=BATCH_SIZE, shuffle=True) 
test_dataset_loader  = DataLoader(testDataset , batch_size=BATCH_SIZE, shuffle=False)
embedding_dim = 5709

class LSTMNetwork(nn.Module):
    def __init__(self, embedding_dim, hidden_dim, num_labels):
        super(LSTMNetwork, self).__init__()
        self.lstm = nn.LSTM(embedding_dim, hidden_dim)
        self.label = nn.Linear(hidden_dim*20, num_labels)
    def forward(self,tweet):
        h_n, c_n = self.lstm(tweet)
        x =self.label(h_n.view(len(tweet),1,-1))
        return x

model = LSTMNetwork(embedding_dim, hidden_dim, vocab_size, num_labels).to(device)
optimizer = optim.SGD(model.parameters(), lr=1e-3)

def train(dataloader, model, loss_fn, optimizer):
    size = len(dataloader.dataset)
    count = 0
    for batch, (X,y) in enumerate(dataloader):
        X, y = X.to(device), y.to(device)
        loss = loss_fn(pred,y)

The result of:

x,y= next(iter(train_dataset_loader))
torch.Size([32, 20, 128])
Out[220]: torch.Size([32])

Am I right in assuming that the 3 dimensions expected are because it’s a 32x3 tensor with the predictions for each class? If so, how do I correct that? Let me know if you need more information.

Thanks again everyone!

Hi Bolanus!

Perhaps the most likely cause of your error would be an unexpected
singleton dimension sneaking into pred.

Less likely would be NLLLoss getting confused by y having the wrong
type (but such a scenario wouldn’t give you the error message you quote
in the latest stable version of pytorch, 1.11.0).

Try printing out the shape and dtype of pred and y just before you
call loss_fn (pred, y). Also, tell us what version of pytorch you are


K. Frank