I have a list of tensors
tw = [tensor([0.7762]), tensor([0.4388, 0.5389]), tensor([-0.4876, -0.2691, 0.0869])] and another tensor
tensor([[-0.0297, -0.3764, 0.4388, 0.5389, 2.0634, -1.5948, -0.2823, -0.2109, 0.7762, 1.8073, -0.9201, -1.7119, 0.5853, -0.7324, -1.2306, 0.3180, -1.2586, -1.5528, -0.4876, -0.4876], [-0.0297, -0.3764, 0.4388, 0.5389, 2.0634, -1.5948, -0.2823, -0.2109, 0.7762, 1.8073, -0.9201, -1.7119, 0.5853, -0.7324, -1.2306, 0.3180, -1.2586, -1.5528, -0.4876, -0.4876], [-0.0297, -0.3764, 0.4388, 0.5389, 2.0634, -1.5948, -0.2823, -0.2109, 0.7762, 1.8073, -0.9201, -1.7119, 0.5853, -0.7324, -1.2306, 0.3180, -1.2586, -1.5528, -0.4876, -0.4876]]).
I checked sent[0][2] == tw[1][0] and the result was tensor(False). I do not understand what is problem here. Both data types are same and when I print both of them separately they show same result tensor(0.4388).
We can’t ensure the two floating type values are the same even though they are displayed as the same value.
I give you a material which can help you understand what’s going on in your code.
Thanks @thecho7 , but when I checks for following example, it works
t = torch.Tensor([[1, 2, 3], [1,2,4]])
t[0][1] == 2
result:
tensor(True)
Why is it so? I read your link and all I understood was it may or may not be same but how can I work around this?
I can convert to integer type as mentioned in the article but I will pay with performance as I will not be able to use GPU in this case (source) .
The reason you got the result tensor(True) is because they are integer.
In any case, it is not recommended that comparing two floating types.
Understood. But
Even if I check t[0][1] == torch.Tensor([2]) result is tensor([True]).
As @thecho7 described: comparing floating point numbers for a bitwise identical representation will fail in most use cases due to the limited floating point precision and you should compare their error against a small eps value defined by the dtype.
As you can see in this example both sums should have the same value, which is not the case due to a different order of operations:
a = torch.randn(100, 100, 100)
s1 = a.sum()
s2 = a.sum(0).sum(0).sum(0)
print((s1 - s2).abs().max())
# tensor(6.1035e-05)