# Static/Dynamic Quantization

**URL:** <https://discuss.pytorch.org/t/static-dynamic-quantization/103980>\
**Category:** quantization\
**Created:** [November 25, 2020, 2:50am UTC](https://discuss.pytorch.org/t/static-dynamic-quantization/103980 "2020-11-25T02:50:39Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mhamdan](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mhamdan/32/29936_2.png) [@mhamdan](https://discuss.pytorch.org/u/mhamdan)\
**Post date:** [November 25, 2020, 2:50am UTC](https://discuss.pytorch.org/t/static-dynamic-quantization/103980/1 "2020-11-25T02:50:39Z")

</div>

I tried quantizing a model using both static and dynamic quantization. Both schemes quantized the weights of the layers but did not quantize the biases. Is there a reason why? and how can I quantize the biases?

Implementation is similar to [this](https://github.com/Sanjana7395/static_quantization/blob/master/quantization%20pytorch.ipynb)

---

<div class="post-metadata">

**Author:** ![dskhudia](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/ebca7d/32.png) [@dskhudia](https://discuss.pytorch.org/u/dskhudia)\
**Post date:** [December 2, 2020, 5:48pm UTC](https://discuss.pytorch.org/t/static-dynamic-quantization/103980/2 "2020-12-02T17:48:04Z")

</div>

biases are not quantized and kept in fp32. For convs and Linears, bias are dynamically quantized before addition while doing convs/Linears.

---

<div class="post-metadata">

**Author:** ![mhamdan](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mhamdan/32/29936_2.png) [@mhamdan](https://discuss.pytorch.org/u/mhamdan)\
**Post date:** [December 2, 2020, 9:55pm UTC](https://discuss.pytorch.org/t/static-dynamic-quantization/103980/3 "2020-12-02T21:55:05Z")

</div>

Hi, Thanks for your reply.  
If you’re saying that biases are not quantized, what do you then mean by biases are dynamically quantized for linears?

---

<div class="post-metadata">

**Author:** ![dskhudia](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/ebca7d/32.png) [@dskhudia](https://discuss.pytorch.org/u/dskhudia)\
**Post date:** [December 3, 2020, 8:10pm UTC](https://discuss.pytorch.org/t/static-dynamic-quantization/103980/4 "2020-12-03T20:10:45Z")

</div>

When the linear is run, it converts biases to int32 before adding to matmul result.

---

<div class="post-metadata">

**Author:** ![mhamdan](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mhamdan/32/29936_2.png) [@mhamdan](https://discuss.pytorch.org/u/mhamdan)\
**Post date:** [December 3, 2020, 8:25pm UTC](https://discuss.pytorch.org/t/static-dynamic-quantization/103980/5 "2020-12-03T20:25:15Z")

</div>

I see. Do you have any idea why PyTorch does it like that?

---

<div class="post-metadata">

**Author:** ![dskhudia](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/ebca7d/32.png) [@dskhudia](https://discuss.pytorch.org/u/dskhudia)\
**Post date:** [December 4, 2020, 2:34am UTC](https://discuss.pytorch.org/t/static-dynamic-quantization/103980/6 "2020-12-04T02:34:38Z")

</div>

if quantized, biases are usually quantized with a scale = activation\_scale \* weight\_scale so that quantized bias can directly be added to matmul output in quantized domain. In pytorch eager mode (due to dynamic nature of pytorch graph), knowing activation scale statically is impossible.