Numerical differences between numpy and pytorch?

Hi,

Unfortunately no :confused:
The reason is that floating point operations are not associative: (a + b) + c != a + (b + c). And so any difference in the order where things are accumulated will lead to such discrepancies.
For these ops, both pytorch and numpy use multithreading. But because they do this in a slightly different way, you see these differences.

Note that some operations are not even deterministic (usually on the GPU) and running it twice in a row won’t give you bit-perfect same results. See the note in the doc about determinism if you want to learn more.