Could not run 'aten::add_.Tensor' with arguments from the 'QuantizedCPU' backend

quantization
1

Hi All,

I was trying to quantize a resnet model. I thought of starting with some toy models, so, instead of coding one myself (also, I am slightly scared with the big model :sweat_smile:) I did the following

class CustomResNet(nn.Module): 

    def __init__(self,num_classes,feature_extract):

        super(CustomResNet, self).__init__()

        self.resnet = models.resnet18(pretrained=False)

        self.resnet.conv1 = nn.Conv2d(in_channels=1, out_channels=64, kernel_size=(7,7), stride=(2,2)) #I am working only with b/w images

        self.resnet.num_classes = num_classes #This is specific to my test case

        set_parameter_requires_grad(self.resnet, feature_extract)

        self.quant = torch.quantization.QuantStub()

        self.dequant = torch.quantization.DeQuantStub()

    

    def forward(self, x):

      x = self.quant(x)

      x = self.resnet(x)

      x = self.dequant(x)

      return x

Then I quantized it and after printing the model, I could see that all the modules have been quantized.
However, when I try to evaluate it using a sample dataset, then, I am getting the following error:

RuntimeError: Could not run 'aten::add_.Tensor' with arguments from the 'QuantizedCPU' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::add_.Tensor' is only available for these backends: [CPU, CUDA, MkldnnCPU, SparseCPU, SparseCUDA, Meta, BackendSelect, Named, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradNestedTensor, UNKNOWN_TENSOR_TYPE_ID, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, Tracer, Autocast, Batched, VmapMode].

I have assumed that this is because the modules have been quantized, but the skip connections in the model and the corresponding operation have not been done.
Can anyone kindly review and let me know, whether this is the correct assumption?

Regards,Arpan

2

Hi Arpan,

The error you are getting is more or less from here:

i.e. that op seems to be for sparse tensors, not quantized ones. Its complaining because it doens’t list QuantizedCPU in the dispatch section of that op. I’m not sure exactly what you are doing for the system to be giving you that error without a full repro since the error doesn’t include the full stack trace. Your code shows the model bus is missing the actual quantization process you are using.

Otherwise, I’d recommend taking another look at the quantization docs/tutorials to make sure you’re doing things in the right order, it can be quite finnicky. That tutorial may be a better place to start and once you understand that, you can apply those techniques to your image classification problem.

tutorials
https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html

you can find the other tutorials under Model Optimization in the sidebar within the pytorch tutorials pages.

1 Like
3

For tensor operations, you will need to use FloatFunctionals to ensure that quantization works correctly. See for example here: vision/resnet.py at master · pytorch/vision · GitHub

1 Like