Set qconfig = None but QDQ still appears

quantization
1

Hi, I’ve set the qconfig = None for the Detect layer of YOLOv5

image
image602×850 72.4 KB

However, when exported to ONNX, the weights still appears to be quantized?

image
image1168×353 48 KB

How do i have the Detect layer be fully non-quantized?

I am doing this as the post processed QAT model does not have weights attached to the Conv2Ds of the Detect layer, which I am guessing is causing further exporting issues? The ‘+’ sign is missing form the weights

image
image975×632 52 KB

Thanks!

2

can you provide more context? how are you quantizing the model exactly, its hard to answer about what’s going wrong without understanding what you are specifically doing.

3

image
image933×269 40.8 KB

  1. i added QDQs in the Conv class
  2. fuse the model [conv bn relu]

image

4

You haven’t quantized the model in what you’ve walked through so far. Is there a convert step somewhere that you haven’t listed?

Can you provide a repro that demonstrates the error?

5

Yup! the actual quantization happens in ONNX. I’m referring to this GitHub - neuralmagic/sparseml: Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models repo for the ONNX quantization process. But that isnt the main issue here

6

Would you have a small reproducible example to demonstrate the issue? It would be hard to debug / provide some recommendation without more context.