How to avoid Quantization warning: "Must run observer before calling calculate_qparams."?

quantization
1

I worked with the pytorch tutorial for static quantization and when running the line:

torch.quantization.convert(per_channel_quantized_model, inplace=True)

I receive the following warning:

.../torch/quantization/observer.py:845: 
UserWarning: must run observer before calling calculate_qparams. Returning default scale and zero point

I call the convert function within the following lines of code:

per_channel_quantized_model = load_model(..)
per_channel_quantized_model.eval()
per_channel_quantized_model.fuse_model()
per_channel_quantized_model.qconfig = torch.quantization.get_default_qconfig('fbgemm')
print(per_channel_quantized_model.qconfig)
torch.quantization.prepare(per_channel_quantized_model, inplace=True)
evaluate(per_channel_quantized_model, ...)
torch.quantization.convert(per_channel_quantized_model, inplace=True)

Does somebody have an idea what the warning means and how I can avoid that? I appreciate any hints and suggestions!

1 Like
2

Facing the same issue. torch.quantization.convert is supposed to run the observers, right. This warning does not make sense.

3

The prepare script inserts the observers. After that when model forward is run it also runs the observers.
If you call convert without calling prepare then can complain about not running observers.

Which model are you running this on? We can take a look if there is a repro.

4

Thanks for your replies, @khizar-anjum and @supriyar!

After @khizar-anjum comments, I also filed a issue on github. The warning is thrown when running the static quantization tutorial. I also received the warning in a SSD-type model I wrote. The quantization lead to a low accuracy and I began asking myself if it was caused by the improper quantization the observer warns against.

1 Like
5

Hello, I also encountered this problem, is there any latest solution, thank you!

6

see here for a solution.

7

Replace self.skip_add.add with torch.add

class InvertedResidual(nn.Module):
def init(self,in_channel,out_channel,stride,expand_ratio):
super(InvertedResidual, self).init()
hidden_channel=int(round(in_channel*expand_ratio))
self.shortcut=stride==1 and in_channel==out_channel

    layers=[]
    if expand_ratio!=1:
        #1x1 pointwise conv
        layers.append(ConvBnRelu(in_channel,hidden_channel,kersize=1))
    layers.extend([
        # 3x3 depthwise conv
        ConvBnRelu(hidden_channel,hidden_channel,stride=stride,groups=hidden_channel),
        nn.Conv2d(hidden_channel,out_channel,kernel_size=1,bias=False),
        nn.BatchNorm2d(out_channel),
    ])

    self.conv=nn.Sequential(*layers)
    #self.skip_add = nn.quantized.FloatFunctional()

def forward(self,x):
    if self.shortcut:
        #return self.skip_add.add(x,self.conv(x))
        return torch.add(x,self.conv(x))
    else:
        return self.conv(x)

RuntimeError: Could not run ‘aten::add.Tensor’ with arguments from the ‘QuantizedCPUTensorId’ backend. ‘aten::add.Tensor’ is only available for these backends: [CPUTensorId, MkldnnCPUTensorId, SparseCPUTensorId, VariableTensorId].

Where did I write it wrong? thanks!

1 Like
8

I am still facing the same issue even after following the instructions present in the link you shared. Are there any further updates to it?