You can pass USE_NNPACK=0 to the setup.py following the instructions to build from source from here.
USE_NNPACK=0
setup.py