Fft operations on Mobile

Mobile
1

I am trying to run audio classification model on Android device, but I am getting error:
RuntimeError: fft: ATen not compiled with MKL support, it’s caused by MelSpectrogram transformation.
I found few related issues on GitHub:

But neither of them had clear solution. What is proper way to fix this problem, preferably with minimal changes to model.

2

Thanks for reaching out. Wondering if there is any detailed log your could share? @iseeyuan @IvanKobzarev could further help you from there.

3

@cccclai may be able to help as well.

4

@Kablys Thanks for raising this up! I don’t think we have a solution on this yet. I’ll follow up in Issue #41592 and see if there is any update there. We are also looking for contributions on implementing those ops in mobile.

5

Simplest PyTorch code to recreate issue:

import torch

def only_stft():
    return torch.stft(torch.tensor([1.0, 1.0, 1.0]), 4)

stft = torch.jit.script(only_stft)
stft.save("stft.pt")

Then I added this Java snippet inside of HelloWorldApp example code: android-demo-app/HelloWorldApp at master · pytorch/android-demo-app · GitHub

    try {
      Module mod = Module.load(assetFilePath(this, "stft.pt"));
      IValue result = mod.forward();
    } catch (IOException e) {
      finish();
    }

The error I get (replaced few personal parts with ...):

Install successfully finished in 4 s 38 ms.
$ adb shell am start -n "org.pytorch.helloworld/org.pytorch.helloworld.MainActivity" -a android.intent.action.MAIN -c android.intent.category.LAUNCHER
Connected to process 23935 on device '...'.
Capturing and displaying logcat messages from application. This behavior can be disabled in the "Logcat output" section of the "Debugger" settings page.
W/orch.helloworld: type=1400 audit(0.0:322903): avc: denied { read } for name="u:object_r:vendor_default_prop:s0" dev="tmpfs" ino=11093 scontext=u:r:untrusted_app_27:s0:c10,c257,c512,c768 tcontext=u:object_r:vendor_default_prop:s0 tclass=file permissive=0
E/libc: Access denied finding property "ro.hardware.chipname"
W/native: [W SpectralOps.cpp:639] Warning: stft will soon require the return_complex parameter be given for real inputs, and will further require that return_complex=True in a future PyTorch release. (function operator())
D/AndroidRuntime: Shutting down VM
E/AndroidRuntime: FATAL EXCEPTION: main
    Process: org.pytorch.helloworld, PID: 23935
    java.lang.RuntimeException: Unable to start activity ComponentInfo{org.pytorch.helloworld/org.pytorch.helloworld.MainActivity}: java.lang.RuntimeException: The following operation failed in the TorchScript interpreter.
    Traceback of TorchScript, serialized code (most recent call last):
      File "code/__torch__.py", line 7, in forward
      def forward(self: __torch__.PlaceholderModule) -> Tensor:
        _0 = torch.tensor([1., 1., 1.], dtype=None, device=None, requires_grad=False)
        _1 = __torch__.torch.functional.stft(_0, 4, None, None, None, True, "reflect", False, None, None, )
             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
        return _1
      File "code/__torch__/torch/functional.py", line 21, in stft
      else:
        input0 = input
      _2 = torch.stft(input0, n_fft, hop_length, win_length, window, normalized, onesided, return_complex)
           ~~~~~~~~~~ <--- HERE
      return _2
    
    Traceback of TorchScript, original code (most recent call last):
      File "C:\Users\...\Anaconda3\envs\...\lib\site-packages\torch\functional.py", line 515, in stft
            input = F.pad(input.view(extended_shape), (pad, pad), pad_mode)
            input = input.view(input.shape[-signal_dim:])
        return _VF.stft(input, n_fft, hop_length, win_length, window,  # type: ignore
               ~~~~~~~~ <--- HERE
                        normalized, onesided, return_complex)
    RuntimeError: fft: ATen not compiled with MKL support
    
        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3534)
        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3689)
        at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:83)
        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:140)
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:100)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2239)
        at android.os.Handler.dispatchMessage(Handler.java:107)
        at android.os.Looper.loop(Looper.java:227)
        at android.app.ActivityThread.main(ActivityThread.java:7822)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1026)
     Caused by: java.lang.RuntimeException: The following operation failed in the TorchScript interpreter.
    Traceback of TorchScript, serialized code (most recent call last):
      File "code/__torch__.py", line 7, in forward
      def forward(self: __torch__.PlaceholderModule) -> Tensor:
        _0 = torch.tensor([1., 1., 1.], dtype=None, device=None, requires_grad=False)
        _1 = __torch__.torch.functional.stft(_0, 4, None, None, None, True, "reflect", False, None, None, )
             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
        return _1
      File "code/__torch__/torch/functional.py", line 21, in stft
      else:
        input0 = input
      _2 = torch.stft(input0, n_fft, hop_length, win_length, window, normalized, onesided, return_complex)
           ~~~~~~~~~~ <--- HERE
      return _2
    
    Traceback of TorchScript, original code (most recent call last):
      File "C:\Users\...\Anaconda3\envs\...\lib\site-packages\torch\functional.py", line 515, in stft
            input = F.pad(input.view(extended_shape), (pad, pad), pad_mode)
            input = input.view(input.shape[-signal_dim:])
        return _VF.stft(input, n_fft, hop_length, win_length, window,  # type: ignore
               ~~~~~~~~ <--- HERE
                        normalized, onesided, return_complex)
    RuntimeError: fft: ATen not compiled with MKL support
    
        at org.pytorch.NativePeer.forward(Native Method)
        at org.pytorch.Module.forward(Module.java:49)
        at org.pytorch.helloworld.MainActivity.onCreate(MainActivity.java:37)
        at android.app.Activity.performCreate(Activity.java:7963)
        at android.app.Activity.performCreate(Activity.java:7952)
        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1306)
E/AndroidRuntime:     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3505)
        	... 11 more
I/Process: Sending signal. PID: 23935 SIG: 9
6

Thanks! That would be helpful for us to look into the issue. We will follow up with you once we get a solution.

7

Hi, so we are hitting the same issue trying to run fft on Android devices. Will this feature be implemented in the near future? It will be appreciated if we have this on PyTorch mobile because it is not easy to find a workaround for fft. Thanks!

8

Bump, same issue here.