Negative label in CrossEntropyLoss

autograd
1 
loss = nn.CrossEntropyLoss()
input = torch.randn(3, 5, requires_grad=True)
target = torch.empty(3, dtype=torch.long).random_(5).long()
target[1] = -100
output = loss(input, target)

The target contains negative values but no error is raised. But if I change target[1] to other negative values, e.g. -1, -10, then the error can be raised correctly. Why is -100 so magic?

System: Ubuntu 18.04
Pytorch version: 1.10.0

2

Hi yzx!

From the documentation for CrossEntropyLoss:

ignore_index=- 100

and

ignore_index (int, optional) – Specifies a target value that is ignored
and does not contribute to the input gradient. When size_average
is True, the loss is averaged over non-ignored targets. Note that
ignore_index is only applicable when the target contains class indices.

That is, -100 is the “magic value” that ignore_index is set to by default, so
those targets are ignored (including the fact that the label would otherwise
be an invalid negative value).

Best.

K. Frank

1 Like