Static quantizing and batch norm error (could not run aten::native_batch_norm with args from QuantCPUTensorid backend')

quantization
1

Hi all,

Working on static quantizing a few models and hitting this error on a basic Resnet18 - any insight into what is missing to complete the quantization?
Did not ‘fuse’ the BN but unclear if that is the core issue?
Any assistance would be appreciated - very hard to find much documentation.

/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in batch_norm(input, running_mean, running_var, weight, bias, training, momentum, eps)
   1921     return torch.batch_norm(
   1922         input, weight, bias, running_mean, running_var,
-> 1923         training, momentum, eps, torch.backends.cudnn.enabled
   1924     )
   1925 
RuntimeError: Could not run 'aten::native_batch_norm' with arguments from the 'QuantizedCPUTensorId' backend. 'aten::native_batch_norm' is only available for these backends: [CPUTensorId, CUDATensorId, MkldnnCPUTensorId, VariableTensorId].

Thanks

2

I think you need to fuse BN since there’s no quantized BN layer.

1 Like
3

Thanks very much - will try to fuse! The docs implied it was more to boost accuracy vs a requirement but makes that it won’t otherwise quantize itself so to speak.

1 Like
4

That was in fact the issue (lack of fusing). Thanks very much @hx89!

1 Like
5

There is no convolutional layer in front, I want to know how to merge the bn layer，thanks!

class MobileFaceNet(Module):
def init(self, embedding_size):
super(MobileFaceNet, self).init()
self.conv1 = Conv_Block(3, 64, kernel=3, stride=2, padding=1)
self.conv2 = Conv_Block(64, 64, kernel=3, stride=1, padding=1, groups=64)
self.conv3 = InvertedResidual(64, 64, kernel=3, stride=2, padding=1, groups=128)
self.conv4 = MakeBlocks(64, num_block=4, kernel=3, stride=1, padding=1, groups=128)
self.conv5 = InvertedResidual(64, 128, kernel=3, stride=2, padding=1, groups=256)
self.conv6 = MakeBlocks(128, num_block=6, kernel=3, stride=1, padding=1, groups=256)
self.conv7 = InvertedResidual(128, 128, kernel=3, stride=2, padding=1, groups=512)
self.conv8 = MakeBlocks(128, num_block=2, kernel=3, stride=1, padding=1, groups=256)
self.conv9 = Conv_Block(128, 512, kernel=1, stride=1, padding=0)
self.conv10 = Conv_Block(512, 512, kernel=7, stride=1, padding=0, groups=512, is_linear=True)
self.ft = Flatten()
self.ln = Linear(512, embedding_size, bias=False)
self.bn = BatchNorm1d(embedding_size)

    self.quant = QuantStub()
    self.dequant = DeQuantStub()

    for m in self.modules():
        if isinstance(m, Conv2d):
            n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
            m.weight.data.normal_(0, math.sqrt(2. / n))
        elif isinstance(m, BatchNorm2d):
            m.weight.data.fill_(1)
            m.bias.data.zero_()

def forward(self, x):
    x = self.quant(x)
    x = self.conv1(x)
    x = self.conv2(x)
    x = self.conv3(x)
    x = self.conv4(x)
    x = self.conv5(x)
    x = self.conv6(x)
    x = self.conv7(x)
    x = self.conv8(x)
    x = self.conv9(x)
    x = self.conv10(x)
    x = self.ft(x)
    x = self.ln(x)
    x = self.bn(x)
    x = self.dequant(x)
    return x

def fuse_model(self):
    for m in self.modules():
        if type(m) == Conv_Block:
            m.fuse_model()