Tutorial on Quantizing Object Detection Model

quantization
1

Hi, I have recently looked at the tutorial for post training static quantization but this is relevant to classifiers. Is there a tutorial/capability to quantize an entire object detection model? If not, what would be the difference if I have a fully trained model and want to quantize only the backbone? Thanks

2

We don’t have a tutorial to quantize a detection model, we can consider it for the future. You can quantize the backbone only as follows, in pseudocode:

# original model (pseudocode)
class M(torch.nn.Module):
  def __init__(self, ...):
    ...
    self.backbone = ...
    self.rpn = ...
    self.head = ...

  def forward(self, x):
    features = self.backbone(x)
    proposals = self.rpn(features)
    head_results = self.head(features, proposals)
    return head_results

# modify M, place quants/dequants to prepare backbone for quantization
class MQuantizeable(torch.nn.Module):
  def __init__(self, ...):
    ...
    self.backbone = ...
    self.rpn = ...
    self.head = ...
  self.quant = torch.quantization.QuantStub()
  self.dequant = torch.quantization.DeQuantStub()

  def forward(self, x):
    # wrap backbone in quant/dequant
    x_quant = self.quant(x)
    features_quant = self.backbone(x)
    features = self.dequant(features_quant)
    proposals = self.rpn(features)
    head_results = self.head(features, proposals)
    return head_results

# quantization call
m = MQuantizeable(...)
# set qconfig on the backbone only
m.backbone.qconfig = ...
m = torch.quantization.prepare(m, ...)
3

When using the above example and converting my model, I get this error:
Traceback (most recent call last):
File “detection/main_mp.py”, line 734, in
main()
File “detection/main_mp.py”, line 592, in main
p = torch.quantization.convert(myModel)
File “/home/megan/.local/lib/python2.7/site-packages/torch/quantization/quantize.py”, line 293, in convert
convert(mod, mapping, inplace=True)
File “/home/megan/.local/lib/python2.7/site-packages/torch/quantization/quantize.py”, line 294, in convert
reassign[name] = swap_module(mod, mapping)
File “/home/megan/.local/lib/python2.7/site-packages/torch/quantization/quantize.py”, line 316, in swap_module
new_mod = mapping[type(mod)].from_float(mod)
File “/home/megan/.local/lib/python2.7/site-packages/torch/nn/quantized/modules/conv.py”, line 243, in from_float
qweight = _quantize_weight(mod.weight.float(), weight_observer)
File “/home/megan/.local/lib/python2.7/site-packages/torch/nn/quantized/modules/utils.py”, line 12, in _quantize_weight
wt_scale.to(torch.double), wt_zp.to(torch.int64), 0, torch.qint8)
RuntimeError: No function is registered for schema aten::quantize_per_channel(Tensor self, Tensor scales, Tensor zero_points, int axis, ScalarType dtype) -> Tensor on tensor type CUDATensorId; available functions are CPUTensorId, VariableTensorId

I have moved my model to cpu and confirmed it is not running on CUDA.

4

The error message is in fact saying that something is on CUDA. Does something like this pass for your model?

def assert_and_get_unique_device(module: torch.nn.Module) -> Any:
    """
    Returns the unique device for a module, or None if no device is found.
    Throws an error if multiple devices are detected.
    """
    devices = {p.device for p in module.parameters()} | \
        {p.device for p in module.buffers()}
    assert len(devices) <= 1, (
        "prepare only works with cpu or single-device CUDA modules, "
        "but got devices {}".format(devices)
    )
    device = next(iter(devices)) if len(devices) > 0 else None
    return device
5

Hi, is it possible to use this quantized implementation of resnet : torchvision quantized resnet as a backbone ?
I tried to use it by changing a little torchvision detection backbone_utils (from torchvision.models.quantization import resnet and add a parameter quantize=true for backbone).

When I try to use this with a pretrained maskrcnn_resnet50 I get an error :
KeyError: 'backbone.body.conv1.bias'

Any idea what I missed/didn’t understand ?

Thanks

6

I have a pytorch quantised model (model.pt), need help to load this model.pt for object detetcion

9

could you start a new question and give more detailis?

10

looks like it’s not loaded correctly, are you sure the pretrain weight matches the model you are loading the weight into?