Tensor operations gives me different outputs

index_separator = 2
all_probs = tensor([[1.7453e-01, 8.2525e-01, 2.1337e-04],
        [7.9364e-01, 2.0613e-01, 2.2616e-04],
        [4.1631e-02, 9.5827e-01, 1.0366e-04],
        [9.2631e-01, 7.3529e-02, 1.6452e-04],
        [9.9914e-01, 8.1824e-04, 4.1946e-05],
        [8.0214e-01, 1.9750e-01, 3.6684e-04],
        [9.9894e-01, 1.0302e-03, 3.3393e-05],
        [9.5733e-02, 9.0413e-01, 1.3462e-04],
        [6.5114e-01, 3.4861e-01, 2.4305e-04],
        [4.2742e-02, 9.5717e-01, 8.7021e-05],
        [6.7704e-01, 3.2274e-01, 2.1942e-04],
        [2.8399e-02, 9.7148e-01, 1.1770e-04],
        [1.0530e-01, 8.9452e-01, 1.7405e-04],
        [7.6687e-02, 9.2317e-01, 1.3761e-04],
        [9.8485e-01, 1.4972e-02, 1.7385e-04],
        [8.1003e-01, 1.8972e-01, 2.4742e-04],
        [9.5397e-01, 4.5884e-02, 1.4194e-04],
        [6.4258e-02, 9.3563e-01, 1.0830e-04],
        [9.7275e-01, 2.7137e-02, 1.1549e-04],
        [9.9825e-01, 1.6978e-03, 5.4864e-05],
        [8.5200e-02, 9.1468e-01, 1.2489e-04],
        [1.1248e-01, 8.8739e-01, 1.3314e-04],
        [9.9868e-01, 1.2721e-03, 4.9347e-05],
        [9.8094e-01, 1.8952e-02, 1.0916e-04],
        [9.9782e-01, 2.0968e-03, 8.1622e-05],
        [9.2653e-01, 7.3261e-02, 2.1026e-04]])

normal_index_sum = torch.sum(all_probs[:, :index_separator], dim=1, keepdim=True)
anomaly_index_sum = torch.sum(all_probs[:, index_separator:], dim=1, keepdim=True)

difference = abs(normal_index_sum - anomaly_index_sum).reshape(-1).tolist(); difference
[0.9995732307434082,
 0.9995477199554443,
 0.9997926354408264,
 0.9996709227561951,
 0.9999160766601562,
 0.9992663264274597,
 0.9999333024024963,
 0.9997307062149048,
 0.9995139241218567,
 0.9998259544372559,
 0.9995611310005188,
 0.999764621257782,
 0.9996519684791565,
 0.9997246861457825,
 0.9996522665023804,
 0.9995051026344299,
 0.9997161030769348,
 0.9997833371162415,
 0.9997689127922058,
 0.9998903274536133,
 0.9997502565383911,
 0.9997337460517883,
 0.9999013543128967,
 0.9997817873954773,
 0.999836802482605,
 0.9995794296264648]

However, if I take the item at index 0 and do the computation 1.7453e-01 + 8.2525e-01 - 2.1337e-04, I get an output of 0.99956663 which is close to difference[0] but not quite. Is there a way to make sure the values returned in both cases are consistent? Is there a way to also track very small changes without the approximation effects seen here?

This small error is expected due to the limited floating point precision. A different order of operation could create these errors and on my system the difference between the manual calculation vs. the difference tensor is ~4e-8.

1 Like