How to extract individual weights after per channel static quantization?

I quantized my CNN network with the static quantization module and the per channel quantization . Here is a part of my quantized network:

Sequential(
  (0): QuantizedConv2d(2, 2, kernel_size=(1, 8), stride=(1, 2), scale=0.43247514963150024, zero_point=62, padding=(0, 3), groups=2)
  (1): QuantizedConvReLU2d(2, 32, kernel_size=(1, 1), stride=(1, 1), scale=0.0933830738067627, zero_point=0)
  (2): Identity()
  (3): Identity()
  (4): Dropout(p=0.5, inplace=False)
)

In the end, I tried to extract the weights and quantization parameters of each convolution kernel.

In   [518]: layers[0].weight()
Out[518]: 
tensor([[[[ 0.5521, -0.4270, -0.9423, -0.8687, -0.4932, -0.3313, -0.3755,
            0.0221]]],

        [[[-0.4360, -0.6763, -0.7154, -0.5980, -0.6372, -0.0447, -0.1733,
           -0.2962]]]], size=(2, 1, 1, 8), dtype=torch.qint8,
       quantization_scheme=torch.per_channel_affine,
       scale=tensor([0.0074, 0.0056], dtype=torch.float64),
       zero_point=tensor([0, 0]), axis=0)

I tried to read the weights, but I got this error:

In [562]: layers[0].weight().data[0,0,0,0]
Traceback (most recent call last):

  File "<ipython-input-562-742a141c2263>", line 1, in <module>
    layers[0].weight().data[0,0,0,0]

RuntimeError: Setting strides is possible only on uniformly quantized tensor

Also, I can get a single scale for the whole layer[0], but I do not know what it is related to compared to the per-channel scale (q_per_channel_scales)?

In [567]: layers[0].scale

Out[567]: 0.43247514963150024
1 Like

OK. It seems like we can use
layer[0].weight().int_repr().data[i,j,l,m]
to get the INT8 representation of the weight entries.

Also,
layer[0].weight().dequantize()
gives the tensor of the weights in FP format to have element-wise access to its contents.

2 Likes

Hi,
I tried adopted your advice and it works well
But I have another question
How can I change the respondent value to the model
I change the value directly like what I do in normal models that will not affect the value of the model’s value
like:

number.int_repr()[0]=number.int_repr()[0]*2
number.int_repr()[0]
# Don't have any change
number.dequantize()=number.dequantize()*2
number.dequantize()
# Don't change too

Thank you