RuntimeError: Module ‘NnapiModule’ has no attribute ‘weights’ (This attribute exists on the Python module, but we failed to convert Python type: ‘list’ to a TorchScript type.)


I am trying to quantize my custom model and apply nnapi.
nnapi prototype “(Prototype) Convert MobileNetV2 to NNAPI — PyTorch Tutorials 1.7.1 documentation

I followed the example provided here.
First, as in mobilenet v2, a quant pair was created, inherited as quantizable_my_model, and the pretrained weight was loaded, and the example was followed.

But nnapi_model = torch.backends._nnapi.prepare.convert_model_to_nnapi(MY_MODEL,MY_TENSOR)
The following error occurred in.

Module'NnapiModule' has no attribute'weights' (This attribute exists on the Python module, but we failed to convert Python type:'list' to a TorchScript type.):
  File "/home/ubuntu/anaconda3/envs/nnapi_2/lib/python3.6/site-packages/torch/backends/_nnapi/", line 35
    def init(self):
        assert self.comp is None
        self.weights = [w.contiguous() for w in self.weights]
                                                ~~~~~~~~~~~~ <--- HERE
        comp = torch.classes._nnapi.Compilation()
        comp.init(self.ser_model, self.weights)
def make_custom_nnapi(output_dir_path):

    example_img = torch.rand(1, 3, 256, 256, device="cpu")
    example_val = torch.rand(1, 10, 2, device="cpu")
    example_jac = torch.rand(1, 10, 2, 2, device="cpu")

    MY_MODEL = quantizable_MY_MODEL() # This function just loads a model that inherits a pair of quantstub and dequantstub. and this function load pretrained weight

    MY_MODEL.fuse_model() # fuse_model() is equal with mobilenetv2

    MY_MODEL.qconfig = torch.quantization.get_default_qconfig('fbgemm')

    MY_MODEL_fp32_prepared = torch.quantization.prepare(MY_MODEL)
    MY_MODEL_int8 = torch.quantization.convert(MY_MODEL_fp32_prepared)
    example_img.nnapi_nhwc = True

    with torch.no_grad():
        MY_MODEL_traced = torch.jit.trace(MY_MODEL_int8, example_img)
    nnapi_model = torch.backends._nnapi.prepare.convert_model_to_nnapi(MY_MODEL_traced,example_img) #error occured this line
class QuantizableMYMODEL(MYMODEL2Q):
    def __init__(self, *args, **kwargs):
        MobileNet V2 main class
           Inherits args from floating point MobileNetV2
        super(QuantizableMYMODEL, self).__init__(*args, **kwargs)
        self.quant = QuantStub()
        self.dequant = DeQuantStub()

    def forward(self, x):
        x = self.quant(x)
        # x = self._forward_impl(x)
        x = self.dequant(x)
        return x


def quantizable_MY_MODEL(pretrained=False, progress=True, quantize=False,**kwargs):
    Constructs a MobileNetV2 architecture from
    `"MobileNetV2: Inverted Residuals and Linear Bottlenecks"
    Note that quantize = True returns a quantized model with 8 bit
    weights. Quantized models only support inference and run on CPUs.
    GPU inference is not yet supported
     pretrained (bool): If True, returns a model pre-trained on ImageNet.
     progress (bool): If True, displays a progress bar of the download to stderr
     quantize(bool): If True, returns a quantized model, else returns a float model

    config_path=PATH TO CONFIG
    checkpoint_path=PATH TO CHEKCPOINT
    checkpoint = torch.load(checkpoint_path, map_location=torch.device("cpu"))

    with open(config_path) as f:
        config = yaml.load(f)

    model = Quantizable_MY_MODEL(
    # _replace_relu(model)

    return model

This is my source code, Hope this helps solve this problem. Thank you.

Don’t mind about the syntax of the uppercase letters

Please help me what to do.

cc @David_Reiss @axitkhurana @Martin_Yuan thoughts?

Hi @Realdr4g0n, sorry for the delay here. Do you have the MY_MODEL being passed to convert_model_to_nnapi? Can you share the torchscript graph if possible? MY_MODEL.graph()