Static quantizing and batch norm error (could not run aten::native_batch_norm with args from QuantCPUTensorid backend')

Hi all,

Working on static quantizing a few models and hitting this error on a basic Resnet18 - any insight into what is missing to complete the quantization?
Did not ‘fuse’ the BN but unclear if that is the core issue?
Any assistance would be appreciated - very hard to find much documentation.

/usr/local/lib/python3.6/dist-packages/torch/nn/ in batch_norm(input, running_mean, running_var, weight, bias, training, momentum, eps)
   1921     return torch.batch_norm(
   1922         input, weight, bias, running_mean, running_var,
-> 1923         training, momentum, eps, torch.backends.cudnn.enabled
   1924     )
RuntimeError: Could not run 'aten::native_batch_norm' with arguments from the 'QuantizedCPUTensorId' backend. 'aten::native_batch_norm' is only available for these backends: [CPUTensorId, CUDATensorId, MkldnnCPUTensorId, VariableTensorId].


I think you need to fuse BN since there’s no quantized BN layer.

1 Like

Thanks very much - will try to fuse! The docs implied it was more to boost accuracy vs a requirement but makes that it won’t otherwise quantize itself so to speak.

1 Like

That was in fact the issue (lack of fusing). Thanks very much @hx89!

1 Like

There is no convolutional layer in front, I want to know how to merge the bn layer,thanks!

class MobileFaceNet(Module):
def init(self, embedding_size):
super(MobileFaceNet, self).init()
self.conv1 = Conv_Block(3, 64, kernel=3, stride=2, padding=1)
self.conv2 = Conv_Block(64, 64, kernel=3, stride=1, padding=1, groups=64)
self.conv3 = InvertedResidual(64, 64, kernel=3, stride=2, padding=1, groups=128)
self.conv4 = MakeBlocks(64, num_block=4, kernel=3, stride=1, padding=1, groups=128)
self.conv5 = InvertedResidual(64, 128, kernel=3, stride=2, padding=1, groups=256)
self.conv6 = MakeBlocks(128, num_block=6, kernel=3, stride=1, padding=1, groups=256)
self.conv7 = InvertedResidual(128, 128, kernel=3, stride=2, padding=1, groups=512)
self.conv8 = MakeBlocks(128, num_block=2, kernel=3, stride=1, padding=1, groups=256)
self.conv9 = Conv_Block(128, 512, kernel=1, stride=1, padding=0)
self.conv10 = Conv_Block(512, 512, kernel=7, stride=1, padding=0, groups=512, is_linear=True)
self.ft = Flatten()
self.ln = Linear(512, embedding_size, bias=False) = BatchNorm1d(embedding_size)

    self.quant = QuantStub()
    self.dequant = DeQuantStub()

    for m in self.modules():
        if isinstance(m, Conv2d):
            n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
  , math.sqrt(2. / n))
        elif isinstance(m, BatchNorm2d):

def forward(self, x):
    x = self.quant(x)
    x = self.conv1(x)
    x = self.conv2(x)
    x = self.conv3(x)
    x = self.conv4(x)
    x = self.conv5(x)
    x = self.conv6(x)
    x = self.conv7(x)
    x = self.conv8(x)
    x = self.conv9(x)
    x = self.conv10(x)
    x = self.ft(x)
    x = self.ln(x)
    x =
    x = self.dequant(x)
    return x

def fuse_model(self):
    for m in self.modules():
        if type(m) == Conv_Block: