Hi, I tried quantization but the cpu I have uses avx and not avx2. So I get this error:
RuntimeError: Didn't find engine for operation quantized::linear_prepack NoQEngine
with this code:
model = torch.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8)
Is there any way to use quantization with a cpu that only supports avx?
Sincerely
You could try to use the qnnpack engine with SSE4_1, but “AVX not AVX2” is generally not terribly well-suported.
Hi, thanks for your answer, can you give an example or a link? Google does not find any result with SSE4_1 
QNNPACK is part of the PyTorch github repo, is also needed on ARM.
I don’t know if SSE support is compiled in by default, but I think it is you best bet.
Best regards
Thomas
according to the error message, looks like you don’t have qengine set correctly, can you set the qengine with torch.backends.quantized.engine = “fbgemm”
here is the info for supported backend for quantization: Quantization — PyTorch master documentation