How do we perform layer wise quantization in pytorch i.e I want to quantize only third and fourth layer, how can i do it?
when we prepare model for quantization using model.prepare all the modules present in the whitelist are quantising . But i didn’t find a way to quantize a single layer. Any kind of help is appreciated?
I am finally able to achieve it by setting different config for the layers i want to quantize, rather than using model.qconfig (which does for all the layers in the model).
For example:
I quantized the first layer by accessing the first layer and assigning it the qconfig.
per_channel_quantized_model.features[0][0].qconfig=torch.quantization.get_default_qconfig(‘fbgemm’)
per_channel_quantized_model.features[0][0].qconfig=torch.quantization.get_default_qconfig(‘fbgemm’)
Hope this helps. And if there is any other efficient approach to achieve this please do let me know.
2 Likes
yeah, this is how we do it in eager mode, we have prototype graph mode that works on torchscript models: https://pytorch.org/tutorials/prototype/graph_mode_static_quantization_tutorial.html which can configure layers with a qconfig_dict. Although we might move away from this soon, but this should still generally work if you need to use it now. Note to use the prototype you will need to use nightly build.
1 Like