How to save the quantized model?

RichardMrLu · January 12, 2018, 9:44am

I used linear quantization, but the quantized model’s size unchanged,It seems that ‘torch.save()’ still save weights in float format…
How to save the quantized weights? I am really appreciate your help.

Hyer_Chen · April 2, 2018, 1:14pm

Have solve the problem? Or any idea to do quantization with pytorch?

RichardMrLu · May 14, 2018, 1:13pm

no… I quantized the model to 2 bit but it is still save in 32bit

dlmacedo · July 13, 2018, 7:17pm

Can tou provide the github link to the code to allow us to help?

indrajitsg · September 13, 2018, 11:59am

I have attempted this and am facing the same issues. I used the approach from the following repo:

github.com

aaron-xichen/pytorch-playground/blob/master/quantize.py

import argparse
from utee import misc, quant, selector
import torch
import torch.backends.cudnn as cudnn
cudnn.benchmark =True
from collections import OrderedDict

parser = argparse.ArgumentParser(description='PyTorch SVHN Example')
parser.add_argument('--type', default='cifar10', help='|'.join(selector.known_models))
parser.add_argument('--quant_method', default='linear', help='linear|minmax|log|tanh')
parser.add_argument('--batch_size', type=int, default=100, help='input batch size for training (default: 64)')
parser.add_argument('--gpu', default=None, help='index of gpus to use')
parser.add_argument('--ngpu', type=int, default=8, help='number of gpus to use')
parser.add_argument('--seed', type=int, default=117, help='random seed (default: 1)')
parser.add_argument('--model_root', default='~/.torch/models/', help='folder to save the model')
parser.add_argument('--data_root', default='/tmp/public_dataset/pytorch/', help='folder to save the model')
parser.add_argument('--logdir', default='log/default', help='folder to save to the log')

parser.add_argument('--input_size', type=int, default=224, help='input size of image')
parser.add_argument('--n_sample', type=int, default=20, help='number of samples to infer the scaling factor')

This file has been truncated. show original

When I try to save the model with torch.save the file size does not show any decrease.

indrajitsg · September 14, 2018, 1:38am

Hi Richard - were you able to quantize your PyTorch models successfully?

raghuramank100 · October 31, 2019, 12:06am

With quantization support in pytorch 1.3, this should work if you follow the flow in the pytorch tutorials for quantization: https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html