How to save the quantized model?

(Richard Mr Lu) #1

I used linear quantization, but the quantized model’s size unchanged,It seems that ‘’ still save weights in float format…
How to save the quantized weights? I am really appreciate your help.

(Hyer Chen) #2

Have solve the problem? Or any idea to do quantization with pytorch?

(Richard Mr Lu) #3

no… I quantized the model to 2 bit but it is still save in 32bit

(David Lopes de Macêdo) #4

Can tou provide the github link to the code to allow us to help?

(Indrajit Sen Gupta) #5

I have attempted this and am facing the same issues. I used the approach from the following repo:

When I try to save the model with the file size does not show any decrease.

(Indrajit Sen Gupta) #6

Hi Richard - were you able to quantize your PyTorch models successfully?