I am using Unet model for semantic segmentation. I pass a batch of images to the model. The model is expected to output 0 or 1 for each pixel of the image (depending upon whether pixel is part of person object or not). 0 is for background, and 1 is for foreground.
I am trying to quantize the Unet model with Pytorch quantization apis (static quantization). The model accuracy is good for FBGEMM config. However, the model outputs all pixels as black pixels (background pixels) for all images. The model output is very high positive for background channel and very high negative for forground channel. (I perform softmax to get final classification of the pixel). The code works fine for FBGEMM. Same code produces very different results for QNNPACK. Is there anything missing for QNNPACK code? I am pasting my code below for reference.
# Static Quantization - FBGEMM/QNNPACK
import torch.quantization as Q
import torch
framework = 'qnnpack' # change to fbgemm for x86 architecture
per_channel_quantized_model = model
per_channel_quantized_model.eval()
per_channel_quantized_model.fuse_model()
per_channel_quantized_model.qconfig = Q.get_default_qconfig(framework)
torch.backends.quantized.engine = framework
print("Preparing . . .")
Q.prepare(per_channel_quantized_model, inplace=True)
print("Running model . . .")
eval_model_for_quantization(per_channel_quantized_model, 'cpu')
print("Converting . . .")
Q.convert(per_channel_quantized_model, inplace=True)
print("***************")
print(per_channel_quantized_model)
print("***************")
print("Checking Accuracy . . .")
accuracy = eval_model_for_quantization(per_channel_quantized_model, 'cpu')
print()
print('Evaluation accuracy after quantization', accuracy)
One issue that I did encounter is that the upsampling layers of Unet use nn.ConvTranspose2d which is not supported for quantization. Hence before this layer, we need to dequantize tensors, apply nn.ConvTranspose2d, and then requantize for subsequent layers. Can this be reason for lower accuracy?
NOTE - I did try with QAT for QNNPACK. However, model output does not change i.e. it gives out all black pixels.
Following is the model output for some inputs. The LHS bracket is target value. Values in RHS are output values for 2 channels (background and foreground). The RHS values are passed to a Softmax function and the final result is obtained (…and to be compared with target)
Now look at following model output for QNNPACK quantization. The output is very different from the unquantized version. In particular, values for all pixels is positive for background channel and negative for foreground channel.
Update - Earlier when I worked on QAT, I was using wrong config. After correcting it, the QAT helps improve the accuracy. However I am still interested in knowing (in case of static quantization for qnnpack config) why the output value for all pixels is positive for background channel and negative for foreground channel.
Some models are more sensitive to quantization than others, and you can try selectively quantizing a part of the model, e.g. keep first conv unquantized, to mitigate the problem for post training static quantization. and typically qat will help in these cases as well.
we are landing https://github.com/pytorch/pytorch/pull/46077 which fixes a bug where some qnnpack activations had incorrect numerical values if the preceding layer was in NHWC. If your model used one of the activations patched by that PR, it might help.
I am also trying to use quantized UNet model (static quantization) for semantic segmentation (10 classes). I am facing issues with the fbgemm configuration itself as mentioned in the forum post. If possible, can you point out where I’m going wrong in my quantization? Meanwhile I’m trying to debug using the link shared by @Vasiliy_Kuznetsov