How to save the quantized model?


(Richard Mr Lu) #1

I used linear quantization, but the quantized model’s size unchanged,It seems that ‘torch.save()’ still save weights in float format…
How to save the quantized weights? I am really appreciate your help.


(Hyer Chen) #2

Have solve the problem? Or any idea to do quantization with pytorch?


(Richard Mr Lu) #3

no… I quantized the model to 2 bit but it is still save in 32bit


(David Lopes de Macêdo) #4

Can tou provide the github link to the code to allow us to help?


(Indrajit Sen Gupta) #5

I have attempted this and am facing the same issues. I used the approach from the following repo:

When I try to save the model with torch.save the file size does not show any decrease.


(Indrajit Sen Gupta) #6

Hi Richard - were you able to quantize your PyTorch models successfully?