Quantization on avx

Hi, I tried quantization but the cpu I have uses avx and not avx2. So I get this error:

RuntimeError: Didn't find engine for operation quantized::linear_prepack NoQEngine

with this code:

model = torch.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8)

Is there any way to use quantization with a cpu that only supports avx?

Sincerely

You could try to use the qnnpack engine with SSE4_1, but “AVX not AVX2” is generally not terribly well-suported.

Hi, thanks for your answer, can you give an example or a link? Google does not find any result with SSE4_1 :slight_smile:

QNNPACK is part of the PyTorch github repo, is also needed on ARM.
I don’t know if SSE support is compiled in by default, but I think it is you best bet.

Best regards

Thomas