I quantized my CNN network with the static quantization module and the per channel quantization . Here is a part of my quantized network:
Sequential(
(0): QuantizedConv2d(2, 2, kernel_size=(1, 8), stride=(1, 2), scale=0.43247514963150024, zero_point=62, padding=(0, 3), groups=2)
(1): QuantizedConvReLU2d(2, 32, kernel_size=(1, 1), stride=(1, 1), scale=0.0933830738067627, zero_point=0)
(2): Identity()
(3): Identity()
(4): Dropout(p=0.5, inplace=False)
)
In the end, I tried to extract the weights and quantization parameters of each convolution kernel.
In [518]: layers[0].weight()
Out[518]:
tensor([[[[ 0.5521, -0.4270, -0.9423, -0.8687, -0.4932, -0.3313, -0.3755,
0.0221]]],
[[[-0.4360, -0.6763, -0.7154, -0.5980, -0.6372, -0.0447, -0.1733,
-0.2962]]]], size=(2, 1, 1, 8), dtype=torch.qint8,
quantization_scheme=torch.per_channel_affine,
scale=tensor([0.0074, 0.0056], dtype=torch.float64),
zero_point=tensor([0, 0]), axis=0)
I tried to read the weights, but I got this error:
In [562]: layers[0].weight().data[0,0,0,0]
Traceback (most recent call last):
File "<ipython-input-562-742a141c2263>", line 1, in <module>
layers[0].weight().data[0,0,0,0]
RuntimeError: Setting strides is possible only on uniformly quantized tensor
Also, I can get a single scale for the whole layer[0], but I do not know what it is related to compared to the per-channel scale (q_per_channel_scales)?
In [567]: layers[0].scale
Out[567]: 0.43247514963150024