Pytorch version 1.3.0
>>> torch.tensor(100000016.0, dtype=float) == torch.tensor(100000015.0, dtype=float)
tensor(False)
Some additional context:
You are creating these “integer” tensors using the default type of torch.float32.
As seen in the Wikipedia article to IEEE FP32, integer numbers >2**26 should be rounded to a multiple of 8.
@Tony-Y is in fact creating these numbers as torch.float64, which can be checked via:
a = torch.tensor(100000016.0, dtype=float)
print(a.type())
> torch.DoubleTensor
which has a higher limit for these rounding errors.
1 Like
As described in the linked Wikipedia page:
"Integers between 2**n and 2**(n+1) round to a multiple of 2**(n-23)"
Since your value is between 2**24 and 2**25, the result will thus be rounded to multiples of 2.
a = 16799999
print(2**24 < a)
> True
print(a < 2**25)
> True
print(torch.tensor(a).float()) # 16799999
> tensor(16800000.) # rounded up
print(torch.tensor(a-1).float()) # 16799998
> tensor(16799998.) # representable
print(torch.tensor(a+2).float()) # 16800001
> tensor(16800000.) # rounded down
print(torch.tensor(a+3).float()) # 16800002
> tensor(16800002.) # representable
Thank you for your reply. With the help of your explanation and wiki link, I have understood now ~