When using the FX Graph mode quantization API for Static Post-Training Quantization for my model, the output seems incorrect. As shown on this page, when displaying the weight differences before and after quantization using ns.compare_weights from Numeric Suite, the histogram for LayerNorm in the float model does not appear (see the attached image). The values of the weights do exist, but since they are not displayed on the graph, it’s difficult to determine if the quantization has been done correctly. Am I missing something?
torch version: 2.1.0
There is an update. After a detailed check, it was found that the weights of LayerNorm are exactly the same in both Float and Quant versions. Despite the layer name changing to QuantizedLayerNorm after quantization, is it correct to understand that the weights of the normalization layer do not get quantized?
The QuantizedLayerNorm module calls into a quantized layer norm op (see here ), which also takes in scale and zero point. So I would expect that the weights are the same for the two layers, but that QuantizedLayerNorm is a bit faster, since it calls into the quantized op instead of the normal one.
