Bug: [W NNPACK.cpp:80] Could not initialize NNPACK! Reason: Unsupported hardware

Hi all,
I trained a model on my ubuntu server using a GPU.
I am now trying to load this model on a python website (using a server that has only a CPU).
I am successfully able to load the model, but when I try to do inference I get an error:

[W NNPACK.cpp:80] Could not initialize NNPACK! Reason: Unsupported hardware.

what can I do to fix this?

Is this really an error (and thus your application is crashing) or just a warning (I assume the latter)?
If I’m not mistaken, NNPACK needs AVX2 CPU instructions, so if these are not supported on your CPU, you would get this warning. You could potentially build PyTorch from source without NNPACK to silence this warning or ignore it.

3 Likes

Hi,

This is a runtime error and it makes the program crash.

How can I build pytorch from source without NNPACK?

Or - is there any other way that you would recommend?

I am trying to use a model that I trained on a server that has GPUs and the current machine doesn’t.

Thank you,

Ariel

You can pass USE_NNPACK=0 to the setup.py following the instructions to build from source from here.

Hey I’m getting the same error running the new M1. I tried following the “From Source” blog, but I couldn’t figure out where I would pass USE_NNPACK=0. I saw something about “export environment variable”, but I’m not sure where to do so. Could you please assist me?

Thanks!

1 Like

You can prepend the env variable to the build command:

USE_NNPACK=0 python setup.py install

or set it via:

export USE_NNPACK=0

before building PyTorch from source.

5 Likes

Hey, thanks a lot for your response! I attempted building from source and after passing git submodule update --init --recursive I passed python setup.py build and got the error:

RuntimeError: no cmake or cmake3 with version >= 3.5.0 found

Passing USE_NNPACK=0 python setup.py install raised the same error!

You may have to install cmake.
On linux, check cmake version using cmake --version.

2 Likes

Genius! Thanks a lot.

Building from source and running the above command now raises this error:

raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command ‘[‘cmake’, ‘–build’, ‘.’, ‘–target’, ‘install’, ‘–config’, ‘Release’, ‘–’, ‘-j’, ‘8’]’ returned non-zero exit status 1.

Could the issue be due to incompatibility with the M1 ARM-based system?

The posted stack trace doesn’t include the error message, so you would need to search a little bit further up to see why the build is failing.

This worked for me, Thanks!!

Hey @ptrblck ,
I’am running in the same error. But mine is looking different and I don’t really know if this is only one error messsage.
My OS:
Kali Linux

My error message:

/home/anaconda3/envs/speech/lib/python3.7/site-packages/torch/functional.py:516: UserWarning: stft will require the return_complex parameter be explicitly  specified in a future PyTorch release. Use return_complex=False  to preserve the current behavior or return_complex=True to return  a complex output. (Triggered internally at  /opt/conda/conda-bld/pytorch_1607370152014/work/aten/src/ATen/native/SpectralOps.cpp:653.)
  normalized, onesided, return_complex)
/home/anaconda3/envs/speech/lib/python3.7/site-packages/torch/functional.py:516: UserWarning: The function torch.rfft is deprecated and will be removed in a future PyTorch release. Use the new torch.fft module functions, instead, by importing torch.fft and calling torch.fft.fft or torch.fft.rfft. (Triggered internally at  /opt/conda/conda-bld/pytorch_1607370152014/work/aten/src/ATen/native/SpectralOps.cpp:590.)
  normalized, onesided, return_complex)
[W NNPACK.cpp:80] Could not initialize NNPACK! Reason: Unsupported hardware.

After this error message is being displayed it is sort of creating an endless loop which is printing out every second a new row. So my program is being completely crashed because of this error message

Is someone able to help me out with that or know what this error message is caused by?
Thank’s for every suggestion and help in advance:)

It seems you are hitting the same issues, so you could also try to rebuild PyTorch without NNPACK or ignore the warning (do you know, if this is a warning or an error?).

@ptrblck I think it’s more like an error because the program is breaking down completely after this message.
Is this one error message? Or are there two?
Because of this:

/home/anaconda3/envs/speech/lib/python3.7/site-packages/torch/functional.py:516: UserWarning: stft will require the return_complex parameter be explicitly  specified in a future PyTorch release. Use return_complex=False  to preserve the current behavior or return_complex=True to return  a complex output. (Triggered internally at  /opt/conda/conda-bld/pytorch_1607370152014/work/aten/src/ATen/native/SpectralOps.cpp:653.)
  normalized, onesided, return_complex)
/home/anaconda3/envs/speech/lib/python3.7/site-packages/torch/functional.py:516: UserWarning: The function torch.rfft is deprecated and will be removed in a future PyTorch release. Use the new torch.fft module functions, instead, by importing torch.fft and calling torch.fft.fft or torch.fft.rfft. (Triggered internally at  /opt/conda/conda-bld/pytorch_1607370152014/work/aten/src/ATen/native/SpectralOps.cpp:590.)
  normalized, onesided, return_complex)

@ptrblck are you still there?

The two new outputs are warnings, so you might want to check them and fix the usage in your code.
If the NNPACK issue is creating an error, please refer to the previous post and try to rebuild PyTorch without NNPACK.

Hey @ptrblck is there any way to install pytorch and right after torchaudio? Becuase when I try to install torchaudio via conda or pip my torch module which I installed from source is being deleted

You could try to use --no-deps while installing torchaudio, but note that pip or conda would only downgrade PyTorch, if torchaudio specifies a specific PyTorch version as a dependency, so your installation might break in case your source build is not compatible with the desired torchaudio version.

A proper approach would be to either build torchaudio also from source or to try to install the nightly releases, which should have relaxed requirements.

Hi @ptrblck, I tried both of these options to set USE_NNPACK=0, but still got the error message as in the title.

For the first option, I did

USE_NNPACK=0 pip3 install --pre torch torchvision torchaudio -f https://download.pytorch.org/whl/nightly/cu111/torch_nightly.html

to install the latest nightly package; for the second, I added

export USE_NNPACK=0;

to right before my previous command.

Is this behavior expected? Thanks!

(This error comes right after the UserWarning that I should use torch.linalg.qr instead of torch.qr. Is it related to the error here?)