Sliced tensor.mean numerical issues

1

I’ve found what seems to be a strange bug in PyTorch 1.5.0 that is absent from 1.7.1.

In 1.5.0 the mean of a large tensor containing large values is incorrect when including more than one column in the slice:

>>> t=torch.randn(1000000,2,1,dtype=torch.float).abs()+1000
>>> t[:,0:1].mean(dim=0), t[:,0:2].mean(dim=0)[0]
(tensor([[1000.2383]]), tensor([1010.0195]))

In 1.7.1 the same code works as expected:

>>> t=torch.randn(1000000,2,1,dtype=torch.float).abs()+1000
>>> t[:,0:1].mean(dim=0), t[:,0:2].mean(dim=0)[0]
(tensor([[1000.7977]]), tensor([1000.7977]))

I’d like to be able to work around this in 1.5.0 if possible. Any ideas what is going on?

2

The errors are most likely created due to the limited floating point precision and the applied algorithm, which might have changed between 1.5.0 and 1.7.1.