Uint64 tensors do not wraparound on overflow

1

Dear experts,

I am seeing that uint64 datatype in torch does not have a wraparound behavior on overflow/underflow (which is the expected behavior in C/C++ from my understanding). Below is my code:

import torch

A = torch.tensor([2 ** 40], dtype=torch.uint64)
B = torch.tensor([2 ** 40], dtype=torch.uint64)
C = A * B
print(C)
A = torch.tensor([2 ** 40], dtype=torch.long)
B = torch.tensor([2 ** 40], dtype=torch.long)
C = A * B
print(C)

Both prints out 0. For torch.long, I see this makes sense as overflow/underflow is usually undefined behaviour. But I am surprised to see this happening for uint64.

Below are some of the questions:

  1. Is this expected (is the unsigned number overflow behavior deviating from C/C++ for torch)?
  2. Is there a way for me to make it wraparound (e.g., by compiling the torch with a certain flag?)
  3. If not, is there any way I can achieve my desired behavior?

I appreciate any pointers in advance.

2

Hi kmaeng!

You are seeing the correct behavior (as do I on pytorch version 2.5.1).

I think you are being fooled by the fact that 2**40 * 2**40 = 2**80 and
2**80 = 0 mod 2**64. So 0 is the correct “wraparound” result.

Consider:

>>> import torch
>>> torch.__version__
'2.5.1'
>>> A = torch.tensor([2 ** 40 - 1], dtype=torch.uint64)
>>> B = torch.tensor([2 ** 40], dtype=torch.uint64)
>>> A * B
tensor([18446742974197923840], dtype=torch.uint64)
>>> 
>>> A = torch.tensor([2 ** 40], dtype=torch.long)
>>> B = torch.tensor([2 ** 40], dtype=torch.long)
>>> A * B
tensor([0])
>>> 
>>> A = torch.tensor([2 ** 40 - 1], dtype=torch.long)
>>> A * B
tensor([-1099511627776])

Best.

K. Frank

3

Oops. Yeah, this was a stupid mistake. I thought I had -1 but not sure where it went. Thanks for letting me know :stuck_out_tongue: