hello.
I am trying to quantize my custom model and apply nnapi.
nnapi prototype “(Prototype) Convert MobileNetV2 to NNAPI — PyTorch Tutorials 1.7.1 documentation ”
I followed the example provided here.
First, as in mobilenet v2, a quant pair was created, inherited as quantizable_my_model, and the pretrained weight was loaded, and the example was followed.
But nnapi_model = torch.backends._nnapi.prepare.convert_model_to_nnapi(MY_MODEL,MY_TENSOR)
The following error occurred in.
RuntimeError:
Module'NnapiModule' has no attribute'weights' (This attribute exists on the Python module, but we failed to convert Python type:'list' to a TorchScript type.):
File "/home/ubuntu/anaconda3/envs/nnapi_2/lib/python3.6/site-packages/torch/backends/_nnapi/prepare.py", line 35
def init(self):
assert self.comp is None
self.weights = [w.contiguous() for w in self.weights]
~~~~~~~~~~~~ <--- HERE
comp = torch.classes._nnapi.Compilation()
comp.init(self.ser_model, self.weights)
def make_custom_nnapi(output_dir_path):
example_img = torch.rand(1, 3, 256, 256, device="cpu")
example_val = torch.rand(1, 10, 2, device="cpu")
example_jac = torch.rand(1, 10, 2, 2, device="cpu")
MY_MODEL = quantizable_MY_MODEL() # This function just loads a model that inherits a pair of quantstub and dequantstub. and this function load pretrained weight
MY_MODEL.fuse_model() # fuse_model() is equal with mobilenetv2
MY_MODEL.qconfig = torch.quantization.get_default_qconfig('fbgemm')
MY_MODEL_fp32_prepared = torch.quantization.prepare(MY_MODEL)
MY_MODEL_fp32_prepared(example_img)
MY_MODEL_int8 = torch.quantization.convert(MY_MODEL_fp32_prepared)
example_img.nnapi_nhwc = True
with torch.no_grad():
MY_MODEL_traced = torch.jit.trace(MY_MODEL_int8, example_img)
nnapi_model = torch.backends._nnapi.prepare.convert_model_to_nnapi(MY_MODEL_traced,example_img) #error occured this line
class QuantizableMYMODEL(MYMODEL2Q):
def __init__(self, *args, **kwargs):
"""
MobileNet V2 main class
Args:
Inherits args from floating point MobileNetV2
"""
super(QuantizableMYMODEL, self).__init__(*args, **kwargs)
self.quant = QuantStub()
self.dequant = DeQuantStub()
def forward(self, x):
x = self.quant(x)
# x = self._forward_impl(x)
x = self.dequant(x)
return x
def quantizable_MY_MODEL(pretrained=False, progress=True, quantize=False,**kwargs):
"""
Constructs a MobileNetV2 architecture from
`"MobileNetV2: Inverted Residuals and Linear Bottlenecks"
<https://arxiv.org/abs/1801.04381>`_.
Note that quantize = True returns a quantized model with 8 bit
weights. Quantized models only support inference and run on CPUs.
GPU inference is not yet supported
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet.
progress (bool): If True, displays a progress bar of the download to stderr
quantize(bool): If True, returns a quantized model, else returns a float model
"""
config_path=PATH TO CONFIG
checkpoint_path=PATH TO CHEKCPOINT
checkpoint = torch.load(checkpoint_path, map_location=torch.device("cpu"))
with open(config_path) as f:
config = yaml.load(f)
model = Quantizable_MY_MODEL(
**config["model_params"]["MYMODEL_params"],
**config["model_params"]["common_params"]
)
model.load_state_dict(checkpoint["MYMODEL"])
model.eval()
# _replace_relu(model)
return model
This is my source code, Hope this helps solve this problem. Thank you.
Don’t mind about the syntax of the uppercase letters
Please help me what to do.