How to use ignore_index in torch.nn.CrossEntropyLoss

nlp
1

Given a sequence: a b c [SEP] d e f,
the corresponding labels is : -1 -1 -1 0 1 0 0.

I want to compute the loss loss_func(logits,labels,ignore_index=-1) to ignore the tokens before [SEP].
And I got a RuntimeError: multi-target not supported at /opt/conda/conda-bld/pytorch_1603729096996.

So I have to change labels to “class indice mode” like 4, but how to ignore the previous tokens when calculating loss.

2

Could you post a minimal, executable code snippet using random tensors in your shapes, which would reproduce the issue?
Based on the error message I would guess you are using nn.CrossEntropyLoss and are passing the target in the wrong shape.