Constant time algorithm for torch.conj


So I just read the torch 1.10 release notes and came across the new constant time conjugate of complex tensors.

I would appreciate if you could help me find any information about this constant time algorithm.
(PyTorch`s tweet about this release also mentions constant time transpose. Knowing about that will be great as well)

Thank you.

Hi Anand! .conj() is now an O(1) operation because all it does is flips a bit on the tensor to indicate conjugation. However, the “materialization” of this lazy conjugation is still O(n).

x = torch.randn(100, dtype=torch.cfloat)
y = x.conj() # O(1)
z = x + y # materialization of conjugation [O(n)] + addition [O(n)]

However, in some operations like matrix multiplication, we can fuse conjugation with the operation because the underlying library that’s being called provides the option to indicate conjugation, and that’s where you’d see a significant improvement in performance (due to fewer reads and writes) in many cases.