RuntimeError: Didn't find engine for operation quantized::conv2d_prepack NoQEngine (operator() at /pytorch/aten/src/ATen/native/quantized/cpu/qconv_prepack.cpp:63)

quantization
1

I tried to run tutorial https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html and faced this problem.

I ran at a supercomputer,cuda 10.1,pytorch 1.4,torchvision 0.5.

total information is

Traceback (most recent call last):
File “q.py”, line 327, in
torch.quantization.convert(myModel,inplace=True)
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 316, in convert
convert(mod, mapping, inplace=True)
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 316, in convert
convert(mod, mapping, inplace=True)
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 317, in convert
reassign[name] = swap_module(mod, mapping)
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 339, in swap_module
new_mod = mapping[type(mod)].from_float(mod)
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/nn/intrinsic/quantized/modules/conv_relu.py”, line 49, in from_float
return super(ConvReLU2d, cls).from_float(mod)
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 257, in from_float
mod.bias is not None, mod.padding_mode)
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/nn/intrinsic/quantized/modules/conv_relu.py”, line 29, in init
padding_mode=padding_mode)
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 187, in init
groups, bias, padding_mode)
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 48, in init
self.set_weight_bias(qweight, bias_float)
File “/SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 195, in set_weight_bias
w, b, self.stride, self.padding, self.dilation, self.groups)
RuntimeError: Didn’t find engine for operation quantized::conv2d_prepack NoQEngine (operator() at /pytorch/aten/src/ATen/native/quantized/cpu/qconv_prepack.cpp:63)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x33 (0x7f414c66d193 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: + 0x1e5c2ba (0x7f414e91c2ba in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch.so)
frame #2: + 0x1e5d083 (0x7f414e91d083 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch.so)
frame #3: + 0x33eb009 (0x7f414feab009 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch.so)
frame #4: + 0x40e8547 (0x7f4150ba8547 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch.so)
frame #5: + 0x6de077 (0x7f4197a4e077 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #6: + 0x6a95c4 (0x7f4197a195c4 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #7: + 0x2961c4 (0x7f41976061c4 in /SISDC_GPFS/Home_SE/songxy-jnu/hejy-jnu/.conda/envs/mjx/lib/python3.6/site-packages/torch/lib/libtorch_python.so)

frame #61: __libc_start_main + 0xf5 (0x7f41abb49445 in /lib64/libc.so.6)

2

This means you didn’t compile fbgemm, can you print torch.backends.quantized.supported_engines?

1 Like
3

Hello, Jerry
I have a similar problem, and this is my code error:

UserWarning: must run observer before calling calculate_qparams. Returning default scale and zero point
Returning default scale and zero point "
Traceback (most recent call last):
File “mobile_test.py”, line 122, in
data_loader,bn_list[0],net_config,device=‘cuda:0’))
File “mobile_test.py”, line 55, in prepare_subnet
torch.quantization.convert(subnet, inplace=True)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 473, in convert
convert_custom_config_dict=convert_custom_config_dict)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 508, in _convert
custom_module_class_mapping)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 508, in _convert
custom_module_class_mapping)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 508, in _convert
custom_module_class_mapping)
[Previous line repeated 3 more times]
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 509, in _convert
reassign[name] = swap_module(mod, mapping, custom_module_class_mapping)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/quantization/quantize.py”, line 534, in swap_module
new_mod = mapping[type(mod)].from_float(mod)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/intrinsic/quantized/modules/conv_relu.py”, line 97, in from_float
return super(ConvReLU2d, cls).from_float(mod)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 418, in from_float
return _ConvNd.from_float(cls, mod)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 220, in from_float
return cls.get_qconv(mod, activation_post_process, weight_post_process)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 191, in get_qconv
mod.bias is not None, mod.padding_mode)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/intrinsic/quantized/modules/conv_relu.py”, line 74, in init
padding_mode=padding_mode)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 376, in init
False, _pair(0), groups, bias, padding_mode)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 74, in _init
self.set_weight_bias(qweight, bias_float)
File “/home/yuantian/.local/lib/python3.6/site-packages/torch/nn/quantized/modules/conv.py”, line 384, in set_weight_bias
w, b, self.stride, self.padding, self.dilation, self.groups)
RuntimeError: Didn’t find engine for operation quantized::conv2d_prepack NoQEngine

I try to print the code

torch.backends.quantized.supported_engines
But the result is [None]
And my system is ubuntu

4

And here is my code:
#fuse model
fuse_model(subnet)

#quantize model
subnet.qconfig = torch.quantization.get_default_qconfig('qnnpack')
torch.quantization.prepare(subnet, inplace=True)
# Calibrate
print(torch.backends.quantized.supported_engines)
torch.quantization.convert(subnet, inplace=True)
#optimize
script_subnet = torch.jit.script(model)
script_subnet_optimized = optimize_for_model(script_model)
5

well qnnpack is disabled by default on x86 systems per the info here: RuntimeError: Didn't find engine for operation quantized::conv_prepack NoQEngine · Issue #29327 · pytorch/pytorch · GitHub

that link also includes a workaround, did you try that? I can see that you posted there as well.

more info here too: The Feature Request of Loading Quantized TorchScript Model on Windows with libtorch · Issue #31684 · pytorch/pytorch · GitHub

if you still can’t solve this, could you open a new issue in the forum and include your env information?

1 Like