Getting error by quantize_fx.prepare_fx

Hi, for quantization of YOLOv5 as pytorch doc suggest, I used this code:

import torch
model= torch.hub.load(‘ultralytics/yolov5’, ‘yolov5s’)

import torch.quantization.quantize_fx as quantize_fx
import copy

post training static quantization

model_to_quantize = copy.deepcopy(model)
qconfig_dict = {"": torch.quantization.get_default_qconfig(‘qnnpack’)}
model_to_quantize.eval()

model_prepared = quantize_fx.prepare_fx(model_to_quantize, qconfig_dict)

model_quantized = quantize_fx.convert_fx(model_prepared)

but this error was raised:

TraceError Traceback (most recent call last)
in ()
7 model_to_quantize.eval()
8 # prepare
----> 9 model_prepared = quantize_fx.prepare_fx(model_to_quantize, qconfig_dict)
10 # calibrate (not shown)
11 # quantize

6 frames
/usr/local/lib/python3.7/dist-packages/torch/fx/proxy.py in to_bool(self, obj)
150 information to the graph node using create_node and can choose to return a value.
151 “”"
→ 152 raise TraceError(‘symbolically traced variables cannot be used as inputs to control flow’)
153
154 @compatibility(is_backward_compatible=True)

TraceError: symbolically traced variables cannot be used as inputs to control flow

what’s the problem? (I write this code in colab)

Hi Rasoul,

It would seem that the model uses dynamic control flow, which is currently not supported in FX in general. For more detail, see torch.fx — PyTorch 1.11.0 documentation. Unfortunately, this means the only way to quantize this model is to use the eager mode API at the moment. Here is an example of how to do that: (beta) Static Quantization with Eager Mode in PyTorch — PyTorch Tutorials 1.11.0+cu102 documentation

Best,
-Andrew

1 Like