How to extract the quantized weight of quantized NN model

I am using Post Training Quantization and try to extract the quantized weight for inference phase, but I failed.
I try to directly use

for weight in quantized_model.state_dict():
np.set_printoptions(suppress=True)
print(weight, “\n”, quantized_model.state_dict()[weight].detach().cpu().clone().numpy())

get “TypeError: NumPy conversion for QuantizedCPUQInt8Type is not supported”

Could give me any advice for extracting the quantized weight from the quantized model?
Thank you very much!!

check out https://pytorch.org/docs/stable/quantization.html#quantized-torch-tensor-operations. Some options:

  1. convert your quantized tensor to floating point with x.dequantize()
  2. get the raw integer values with x.int_repr(), this should be used together with x.q_scale() and x.q_zero_point.
1 Like

the error message shows that the weight is already a quantized weight, it’s just numpy operation is not supported in quantized weight, so if you remove the numpy() call it should work.

1 Like