Pretrained quantized models' export to ONNX fails

quantization
1

I am trying to export the pretrained quantized models to ONNX, but it fails. I’ve tried for GoogleNet, ResNet18 and Mobilenet v2, but none of those exported.

I got the following error for GoogleNet and ResNet18

RuntimeError: self.qscheme() == at::kPerTensorAffine INTERNAL ASSERT FAILED at /pytorch/aten/src/ATen/native/quantized/QTensor.cpp:162, please report a bug to PyTorch. clone for quantized Tensor only works for PerTensorAffine scheme right now

3

did you find a solution for this?

4

Hey, i’ve changed qconfig to torch.quantization.get_default_qconfig(‘qnnpack’) and error RuntimeError: self.qscheme() == at::kPerTensorAffine INTERNAL ASSERT FAILED is not an issue any more. This is just workaround.

Now i have an error KeyError: ‘conv2d_relu’ after small changed.

I fused layers for static quantization and it seems onnx doesn’t support fused layers in this format. https://github.com/onnx/onnx/blob/master/docs/Operators.md#Conv

This key doesn’t exist in symbolic_opset (any) so i don’t expect it would work.

5

We probably need a patch to support clone for per channel quantized tensor: https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/native/quantized/QTensor.cpp#L160

6

created an issue: https://github.com/pytorch/pytorch/issues/33309
I’ll fix it when I have time, but feel free to submit a PR as well and I’m happy to review it.