Not Getting an Expected Error

1

I am trying a two-way classification using a linear layer: nn.Linear(4096, 2) to get the final predictions; and deliberately using target from 50 classes. I think I should be getting an error when using CrossEntropy loss, since target has classes outside of final linear layer. However, I am not getting the any error. I am using version 1.5.0. Any insights? Am I missing something?

Output (batch-size of 5):

Target:  tensor([ 6, 12,  8, 29,  3], device='cuda:0')
Pred cls:  tensor([[ 5.7777, 11.3648],
        [ 0.4722,  2.3781],
        [ 2.2436, -1.5575],
        [-4.2130,  1.2372],
        [ 9.2907,  4.8596]], device='cuda:0', grad_fn=<AddmmBackward>)
CE loss:  tensor(1.0909, device='cuda:0', grad_fn=<NllLossBackward>)
2

Hi,

For performance reasons, we cannot do as many checks on the cuda side as we can do on the cpu side :confused:
Running this on the CPU should raise a nice error.
Also I think we improved the GPU behavior in master so you can try with a nightly build.

3

Yes, that’s correct – in CPU mode, it raises an error. If I remember correctly, in version 1.0.0, GPU mode was also raising error, that’s why I was a little concerned. Thank you very much!

4

PyTorch 1.5.0 had a bug where device assert statements were ignored. It’s fixed in 1.6.0 and later, so I recommend to use the latest version.

1 Like