Fft performance

I’m comparing the performance of rfft in torch, numpy and scipy.
The result is that, scipy is 1 time slower than torch, and numpy is 1time slower than scipy.
That is strange. So I did another test. And here is the code:

    import torch
    import numpy as np
    import scipy
    data = np.random.random((10000000,)).astype(np.float32)
    print(data)
    print(data.dtype)
    lp_torch = torch.fft.rfftfreq(10000000, d=1./250000000) < 1000000
    lp_scipy = scipy.fft.rfftfreq(10000000, d=1./250000000) < 1000000
    lp_np = np.fft.rfftfreq(10000000, d=1./250000000) < 1000000
    print('lp:')
    print(lp_torch)
    print(lp_scipy)
    print(lp_np)
    fft_torch = torch.fft.rfft(torch.tensor(data))
    fft_scipy = scipy.fft.rfft(data)
    fft_np = np.fft.rfft(data)
    print('fft:')
    print(fft_torch)
    print(fft_scipy)
    print(fft_np)
    print(fft_torch.dtype)
    print(fft_scipy.dtype)
    print(fft_np.dtype)
    pass_torch = torch.multiply(fft_torch, lp_torch)
    pass_scipy = scipy.multiply(fft_scipy, lp_scipy)
    pass_np = np.multiply(fft_np, lp_np)
    print('pass:')
    print(pass_torch)
    print(pass_scipy)
    print(pass_np)
    print(pass_torch.dtype)
    print(pass_scipy.dtype)
    print(pass_np.dtype)
    pass_torch = torch.fft.irfft(pass_torch)
    pass_scipy = scipy.fft.irfft(pass_scipy)
    pass_np = np.fft.irfft(pass_np)
    print('irfft:')
    print(pass_torch)
    print(pass_scipy)
    print(pass_np)
    print(pass_torch.dtype)
    print(pass_scipy.dtype)
    print(pass_np.dtype)

And the output:

[0.3066935  0.21171719 0.8174513  ... 0.88327384 0.20714097 0.36953172]
float32
lp:
tensor([ True,  True,  True,  ..., False, False, False])
[ True  True  True ... False False False]
[ True  True  True ... False False False]
fft:
tensor([ 5.0000e+06+0.0000j,  1.0524e+03-513.6312j,  3.9784e+02-411.0908j,
         ..., -3.0829e+02-346.9896j,  1.0980e+03+240.6152j,
        -9.4525e+02+0.0000j])
[ 5.0000470e+06  +0.j       1.0524075e+03-513.63116j
  3.9784344e+02-411.09076j ... -3.0828790e+02-346.98987j
  1.0979875e+03+240.6151j  -9.4500000e+02  +0.j     ]
[ 5.00004689e+06  +0.j          1.05240748e+03-513.63097737j
  3.97843491e+02-411.09077203j ... -3.08287811e+02-346.98967654j
  1.09798757e+03+240.61525509j -9.45161769e+02  +0.j        ]
torch.complex64
complex64
complex128
pass:
tensor([5.0000e+06+0.0000j, 1.0524e+03-513.6312j, 3.9784e+02-411.0908j,
         ..., 0.0000e+00-0.0000j, 0.0000e+00+0.0000j,
        -0.0000e+00+0.0000j])
[ 5.0000470e+06  +0.j       1.0524075e+03-513.63116j
  3.9784344e+02-411.09076j ...  0.0000000e+00  -0.j
  0.0000000e+00  +0.j      -0.0000000e+00  +0.j     ]
[ 5.00004689e+06  +0.j          1.05240748e+03-513.63097737j
  3.97843491e+02-411.09077203j ...  0.00000000e+00  -0.j
  0.00000000e+00  +0.j         -0.00000000e+00  +0.j        ]
torch.complex64
complex64
complex128
irfft:
tensor([0.4760, 0.4755, 0.4751,  ..., 0.4774, 0.4769, 0.4764])
[0.47597715 0.47552222 0.47507197 ... 0.4773674  0.4769001  0.47643661]
[0.47597718 0.47552222 0.47507193 ... 0.47736738 0.47690002 0.47643654]
torch.float32
float32
float64

I noticed that, the precision of their results are different. Numpy has the highest precision, and, although torch and scipy are all complex64, their results are still different.

So my question is: does torch do a float16 calculation during rfft?

I’m using torch 1.13.

no, torch doesn’t do float16 calculations during rfft when your tensor is float32.
the way they pad and stuff might be different between libraries, which might explain the differences.