2D tensor multiplication with its transpose - shouldn't it return a symmetric 2D tesnor?

1

I am multiplying a large 2D tensor with its transpose. I am expecting that I would get a symmetric matrix. Here is the sample code.

import torch

n = torch.randn(200, 385)
s = n @ n.T
print(torch.allclose(s, s.T))

This prints False.

Is this expected behavior? Shouldn’t it return True? What am I missing here? Is this a bug?

2

Seems like an issue with numerical accuracy. If this is a problem at some point in your code you can either

  1. Increase numerical precision, i.e., in this case n = torch.randn(200, 385, dtype=torch.float64).
  2. or do the equality check with higher tolerance, e.g., torch.allclose(s, s.T, atol=1e-4). Note that here the default is atol=1e-8 and the “a” stands for tolerance with respect to absolute deviation.
3

Thank you. I was hoping that if I used default type for tensor and default values when I call allclose(), the behaviors would be coherent.

1 Like