How to quantize Mobilenet_V2 after training

I trained mobilenet v2 with output is 16 classes and save model. After I load model and quantize that model, then get error when predict. Can you help me fix this bug. thanks a lot

Code load model after training

def load_model(model_path, num_classes=16, device=torch.device('cpu')):
    model = models.mobilenet_v2()
    n_input_features = model.classifier[1].in_features
    model.classifier[1] = nn.Linear(n_input_features, num_classes)
    model.load_state_dict(torch.load(model_path, map_location=device))
    return model

Code quantize model

model_fp32 = load_model("").to("cpu")
backend = "fbgemm"
model_fp32.qconfig = torch.quantization.get_default_qconfig(backend)
torch.backends.quantized.engine = backend
model_static_quantized = torch.quantization.prepare(model_fp32, inplace=False)
model_static_quantized = torch.quantization.convert(model_static_quantized, inplace=False)

Quantize success

def print_model_size(mdl):, "")
    print("%.2f MB" %(os.path.getsize("")/1e6))


Then I run and get Error

return ops.quantized.conv2d(
NotImplementedError: Could not run '' with arguments from the 'CPU' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit for possible resolutions. '' is only available for these backends: [QuantizedCPU, BackendSelect, Python, Named, Conjugate, Negative, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradLazy, AutogradXPU, AutogradMLC, Tracer, UNKNOWN_TENSOR_TYPE_ID, Autocast, Batched, VmapMode].```

The error message means that the input needs to be quantized.
The easiest way to achieve this might be to wrap the model into a larger model and add torch.quantization.QuantStub/DeQuantStub layers before calling into the model.

The quantization tutorial has these steps, though they are easy to miss.

Best regards


Thanks @tom . But I don’t know how to quantize my model Mobilenetv2 finetuned with output is 16 classes. Can you help me!

Something like

import torch
import torchvision
model = torchvision.models.mobilenet_v2()
# load you weights here into model
qmodel = torch.quantization.QuantWrapper(model)


backend = "fbgemm"
qmodel.qconfig = torch.quantization.get_default_qconfig(backend)
torch.backends.quantized.engine = backend
model_static_quantized = torch.quantization.prepare(qmodel, inplace=False)

# feed real inputs here
model_static_quantized(torch.randn(1, 3, 224, 224))

model_static_quantized = torch.quantization.convert(model_static_quantized, inplace=False)


seems to work for me.

Best regards