I have noticed that the results of
torch.sum and the addition operator are not always equal.
import torch def f(): a = torch.rand(100).view(10,10) s1 = torch.sum(a, dim=0) s2 = torch.zeros(10) for i in range(10): s2 += a[i] assert (s1 == s2).all() for i in range(1000): f()
Running the above code results in assertion failure. If I check the difference between
s2, they are generally of the order of
Similar numpy code however seems to not have this issue.
import numpy as np def f(): a = np.random.rand(100).astype(np.float32).reshape((10,10)) s1 = np.sum(a, axis=0) s2 = np.zeros(10, dtype=np.float32) for i in range(10): s2 += a[i] assert (s1 == s2).all() for i in range(1000): f()
I have run this several times and haven’t had any assertion errors so far.
Is this behavior simply because of floating point precision or is there something else that is causing this issue?