Very Bad Bounding Boxes After Quantization of Model

quantization
1

Hi Folks,
I have been trying to perform static quantization of a model for scene text detection and reduce the precision of the model from float32 to integer8.

I followed the steps mentioned in the official tutorial - fused the conv,batch norm and relu layers ; converted to int ; replaced concat with nn.quantized.Floattensor().
The original model works well, but the prediction from quantized model is really bad -

Prediction from FP32 model -
image

Prediction from INT8 model -
image

Not sure where I’m going wrong :confused:

Would appreciate on any suggestions where I’m going wrong during quantization.

The code is available here - https://github.com/raghavgurbaxani/experiments/blob/master/try_static_quant.py

2

Not sure how it might help you, but you can try using Intel’s Openvino for quantisation.

3

yeah pytorch quantization didn’t yield very good results. Might give OpenVino or Tensortt a try.

4

Seems it’s the same topic as another thread where we are helping on debugging: