Is qunatization supported/possible on Windows ?(getting RuntimeError: Quantized backend not supported )

1

Hello everyone, hope you are having a great time .
I’m trying to get the samples (Tutorials on quantization) to run on windows instead of linux.
for Linux it works just fine, but for windows for example doing simply :

model_bk = models.mobilenet_v2(pretrained=True, progress=True, quantize=True)
num_ftrs=model_bk.classifier[1].in_features

results in :

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
d:\Codes\Quantization\quantization.py in 
----> 1 model_bk = models.mobilenet_v2(pretrained=True, progress=True, quantize=True)
      2 num_ftrs=model_mbnet_fe.classifier[1].in_features
      3 

~\Anaconda3\lib\site-packages\torchvision\models\quantization\mobilenet.py in mobilenet_v2(pretrained, progress, quantize, **kwargs)
     78         # TODO use pretrained as a string to specify the backend
     79         backend = 'qnnpack'
---> 80         quantize_model(model, backend)
     81     else:
     82         assert pretrained in [True, False]

~\Anaconda3\lib\site-packages\torchvision\models\quantization\utils.py in quantize_model(model, backend)
     20     _dummy_input_data = torch.rand(1, 3, 299, 299)
     21     if backend not in torch.backends.quantized.supported_engines:
---> 22         raise RuntimeError("Quantized backend not supported ")
     23     torch.backends.quantized.engine = backend
     24     model.eval()

RuntimeError: Quantized backend not supported

doing :

print(*torch.__config__.show().split('\n'),sep='\n')

results in :

PyTorch built with:

  • MSVC 191125547
  • Intel(R) Math Kernel Library Version 2019.0.4 Product Build 20190411 for Intel(R) 64 architecture applications
  • OpenMP 200203
  • Build settings: BLAS=MKL, BUILD_NAMEDTENSOR=OFF, BUILD_TYPE=Release, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /w /EHa /MP /bigobj -openmp, DISABLE_NUMA=1, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, USE_CUDA=0, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF,

which clearly shows USE_NNPACK=OFF, as nnpack is not even supported on Windows.
So knowing that MKDNN is supported , should it not have been the chosen backend here? (instead of nnpack) ?
Or is there something else that I dont know of, which is needed to get this to work on windows!
Also I noticed torch.backends.quantized.supported_engines returns [none]!

Any help is greatly appreciated
Thank you all in advance

2

See https://github.com/pytorch/pytorch/issues/29327 and https://github.com/pytorch/pytorch/issues/31684#issuecomment-572584403.

1 Like
3

Thanks a lot . that was a refereshing news really! specially knowing the fbgemm is now supported on windows. do you think this can make it into the nighyly build in a near future ? like in the next month or even sooner?

4

It depends on when the fbgemm team thinks it’s ready.

5

Thanks a lot @peterjc123 really appreciate all your hard works on the windows front

6

@peterjc123 Do you mind if I ask a question?
Does MKLDNN need to be supported for the quantization to work as well, or do they just need to add support for qnnpack or fbgemm for windows regardless of the existance of MKDNN support?

7

I believe MKLDNN is not required for quantization and building FBGEMM.

1 Like