IndexError tensors used as indices must be long, byte or bool tensors

1

When I train my model, I always randomly get the following error:

IndexError: tensors used as indices must be long, byte or bool tensors.

I know what this error message saying about but why this error comes out randomly? And should I add .long() whenever trying to index a tensor by another tensor?

1 Like
2

If this error is thrown randomly, the index tensor is most likely changing the type “randomly” in your code base.
I would recommend to narrow down, why the index tensor changes the type and make sure you are using valid indices.

3

Thanks, Since this error is thrown randomly, the program runs normally for some iterations until raise this error, it is really hard to find where does the index tensor get changed. And I think I haven’t change the type of index tensor explicitly, do you know which operations in pytorch might change the index tensor implicitly?

Thank you so much.

4

It’s hard to tell and maybe you could find some similar operations using the examples in the type promotion docs.

5

Thanks ! I found that some times my index tensor has type torch.int32. BTW, why PyTorch doesn’t allow to use torch.int32 type as the indices tensor?

2 Likes
6

May be it is not really random and some of the intermediate operations on data might return back different tensor dtypes.