Tutorial on Quantizing Object Detection Model

Hi, I have recently looked at the tutorial for post training static quantization but this is relevant to classifiers. Is there a tutorial/capability to quantize an entire object detection model? If not, what would be the difference if I have a fully trained model and want to quantize only the backbone? Thanks

We don’t have a tutorial to quantize a detection model, we can consider it for the future. You can quantize the backbone only as follows, in pseudocode:

# original model (pseudocode)
class M(torch.nn.Module):
  def __init__(self, ...):
    self.backbone = ...
    self.rpn = ...
    self.head = ...

  def forward(self, x):
    features = self.backbone(x)
    proposals = self.rpn(features)
    head_results = self.head(features, proposals)
    return head_results
# modify M, place quants/dequants to prepare backbone for quantization
class MQuantizeable(torch.nn.Module):
  def __init__(self, ...):
    self.backbone = ...
    self.rpn = ...
    self.head = ...
  self.quant = torch.quantization.QuantStub()
  self.dequant = torch.quantization.DeQuantStub()

  def forward(self, x):
    # wrap backbone in quant/dequant
    x_quant = self.quant(x)
    features_quant = self.backbone(x)
    features = self.dequant(features_quant)
    proposals = self.rpn(features)
    head_results = self.head(features, proposals)
    return head_results
# quantization call
m = MQuantizeable(...)
# set qconfig on the backbone only
m.backbone.qconfig = ...
m = torch.quantization.prepare(m, ...)

When using the above example and converting my model, I get this error:
Traceback (most recent call last):
File “detection/main_mp.py”, line 734, in
File “detection/main_mp.py”, line 592, in main
p = torch.quantization.convert(myModel)
File “/home/megan/.local/lib/python2.7/site-packages/torch/quantization/quantize.py”, line 293, in convert
convert(mod, mapping, inplace=True)
File “/home/megan/.local/lib/python2.7/site-packages/torch/quantization/quantize.py”, line 294, in convert
reassign[name] = swap_module(mod, mapping)
File “/home/megan/.local/lib/python2.7/site-packages/torch/quantization/quantize.py”, line 316, in swap_module
new_mod = mapping[type(mod)].from_float(mod)
File “/home/megan/.local/lib/python2.7/site-packages/torch/nn/quantized/modules/conv.py”, line 243, in from_float
qweight = _quantize_weight(mod.weight.float(), weight_observer)
File “/home/megan/.local/lib/python2.7/site-packages/torch/nn/quantized/modules/utils.py”, line 12, in _quantize_weight
wt_scale.to(torch.double), wt_zp.to(torch.int64), 0, torch.qint8)
RuntimeError: No function is registered for schema aten::quantize_per_channel(Tensor self, Tensor scales, Tensor zero_points, int axis, ScalarType dtype) -> Tensor on tensor type CUDATensorId; available functions are CPUTensorId, VariableTensorId

I have moved my model to cpu and confirmed it is not running on CUDA.

The error message is in fact saying that something is on CUDA. Does something like this pass for your model?

def assert_and_get_unique_device(module: torch.nn.Module) -> Any:
    Returns the unique device for a module, or None if no device is found.
    Throws an error if multiple devices are detected.
    devices = {p.device for p in module.parameters()} | \
        {p.device for p in module.buffers()}
    assert len(devices) <= 1, (
        "prepare only works with cpu or single-device CUDA modules, "
        "but got devices {}".format(devices)
    device = next(iter(devices)) if len(devices) > 0 else None
    return device

Hi, is it possible to use this quantized implementation of resnet : torchvision quantized resnet as a backbone ?
I tried to use it by changing a little torchvision detection backbone_utils (from torchvision.models.quantization import resnet and add a parameter quantize=true for backbone).

When I try to use this with a pretrained maskrcnn_resnet50 I get an error :
KeyError: 'backbone.body.conv1.bias'

Any idea what I missed/didn’t understand ?


I have a pytorch quantised model (model.pt), need help to load this model.pt for object detetcion

could you start a new question and give more detailis?

looks like it’s not loaded correctly, are you sure the pretrain weight matches the model you are loading the weight into?