Nanton
August 16, 2022, 4:49pm
1
Hi there,
If there is a model with CNN as backbone, LSTM as its head, how to quantize this whole model with post training quantization? It seems we can apply static quantization to CNN and dynamic quantization to LSTM( Quantization β PyTorch 1.12 documentation ). But not very sure how to deal with cases like above one.
Thanks in advance!
1 Like
Hi @Nanton ,
What you said is correct, we have official support for static quantization for CNNs and dynamic quantization for LSTMs.
There is an unreleased prototype of static quantization of LSTMs, an example of how to use it is here: pytorch/test_quantized_op.py at a9ba3fe1dbf2cea45c9a7e723010c27c211f7fe3 Β· pytorch/pytorch Β· GitHub . There is no documentation or tutorial on this feature yet, but we hope to get to it in the future.
1 Like
Nanton
August 18, 2022, 6:20pm
3
Hi @Vasiliy_Kuznetsov ,
Appreciate your reply. I look forward to this feature as well.
Based on what we have now, I was wondering how to quantize a model with both CNN and LSTM . Is there any tutorial available?
Thanks
jerryzh168
(Jerry Zhang)
August 25, 2022, 11:17pm
4
Nanton:
If there is a model with CNN as backbone, LSTM as its head, how to quantize this whole model with post training quantization? It seems we can apply static quantization to CNN and dynamic quantization to LSTM( Quantization β PyTorch 1.12 documentation ). But not very sure how to deal with cases like above one.
we donβt have a tutorial yet. Are you using eager mode quantization? if so LSTM is supported by default, you can follow the original flow: (beta) Static Quantization with Eager Mode in PyTorch β PyTorch Tutorials 1.12.1+cu102 documentation
fx graph mode quantization is not fully supporting static quantization for LSTM yet I think
Simply donβt quantize LSTM part . try search quantize specific layer in pytorch.