Categorical Cross Entropy Cost works with wrong labels

1

Today I received a simple binary class dataset. After training some models I tried to change the labels to one-hot. There I realized that some labels where wrong.

For instance, the numpy.array has the next values (in format int64):
`

        -9223372036854775808 (obviously wrong label)

        0

        1

`

That when converted to torch tensor using torch.from_numpy results in:

`

     -9.2234e+18 (wrong label.)

      0

      1

`

Even more strange is that I can minimize the categorical cross entropy which normally throws error if the labels provided are not in [0,C-1].

I made some checkings. For instance if I put the tags from both classes to lets say -1 and -2 I get error:

    Assertion `t >= 0 && t < n_classes` failed.

If I put the label one to 1 and the other to -1 it throws error. However If I put one label to 1 and the other to -9223372036854775808 everything works perfectly.

What is happening?

1 Like
2

I have tried these with a custom dataset inserting that label and also happens…

3

This seems like a bug.
Could you create a Github issue and post this code to reproduce the issue:

output = torch.randn(5, 2, requires_grad=True)
target = torch.empty(5, dtype=torch.long).random_(2)
target[0] = -9223372036854775808

criterion = nn.CrossEntropyLoss()
loss = criterion(output, target)
print(loss)
4

yes it seems.

Yesterday I show it to a colleague and he realized that this number is the minimum int64 number you can store in a machine

     import numpy
     print numpy.iinfo(numpy.int64)

Probably pytorch check the value of the tags at a C level because if one does:

       numpy.iinfo(numpy.int64).min>=0

That gives you false, not true. We were discussing that at a hardware level this kind of extreme numbers is not processed as other numbers for speed.

I will open the issue

1 Like