Unexpected model output

nlp
1

Hello,
I have a multi classification model ( with 5 classes). The model looks like this:

classifier(
  (embedding): Embedding(28564, 100)
  (lstm): LSTM(100, 32, num_layers=2, batch_first=True, dropout=0.2, bidirectional=True)
  (fc): Linear(in_features=64, out_features=5, bias=True)
  (act): Sigmoid()
)

But when I do predict to a new sampel, I have an output like this:

tensor([[ 4.9442, -4.3852,  0.8900, -2.8472, -4.5958]], device='cuda:0',
       grad_fn=<SliceBackward>)

What should I modify to have an output as a label (0,1,2,3,4)?
Thanks in advance.

2

I assume you are working on a multi-label classification, where each sample might belong to zero, one, or multiple classes?
If so, you could use a threshold and get the predicted label for an active/inactive class as:

out = ...
preds = out > 0.5
# preds would now indicate which class is active (1) or inactive (0) for the sample

In your code snippet your model uses nn.Sigmoid as the last layer, which doesn’t fit the output as it’s negative.
In case you are indeed using nn.Sigmoid, a default threshold of 0.5 could be a good starting point. On the other hand, if your model outputs logits, you could use 0.0 as the default threshold.

3

Thanks, ptrblck, for your reply.
No, I’m working on multiclass classification where there is no overlap between the classes.
In my case, what do you suggest?
Also, I can not use softmax since my loss function is CrossEntropyLoss which has built-in softmax (as I understand from the documentation of pytorch).

Thanks,

4

Ah OK, in that case remove the nn.Sigmoid and pass the raw logits in the shape [batch_size, nb_classes] to nn.CrossEntropyLoss. The targets should have the shape [batch_size] and contain class indices in the range [0, nb_classes-1]. To get the predicted class from the model output use preds = torch.argmax(output, dim=1).