TypeError: quantized_add() missing 2 required positional arguments: 'op_scale' and 'op_zero_point'

quantization
1

Hi I’m trying to export the quantized model to onnx. But im facing the following error while doing so:
TypeError: quantized_add() missing 2 required positional arguments: ‘op_scale’ and ‘op_zero_point’

Here is my skeleton code:

model = # my_model
model.eval()
from torch.quantization import quantize_fx
qconfig_dict = {"": torch.quantization.get_default_qconfig("qnnpack")}

model_prepared = quantize_fx.prepare_fx(model, qconfig_dict,(image_tensor[None],))

model_prepared.eval()
with torch.no_grad():
    for i in range(0,len(eval_ds)):
        image_tensor = eval_ds[i]
        model_prepared(image_tensor[None])

model_int8 = torch.quantization.quantize_fx.convert_fx(model_prepared)
torch.onnx.export(model_int8,image_tensor[None],"onnx_model.onnx",verbose=True,opset_version=15)

Can anyone please help me out with this?
Thanks

2

at what line do you see the error? If you share a stack trace that can help narrow it down.

Does convert_fx produce a valid quantized model in this case?

3

The convert_fx produces a GraphModule, which i presume is a valid model. please let me know if there is any other method to verify it.

The error occurs on the onnx export. Here is the stack trace:

Traceback (most recent call last):
  File "/home/econsy/projects/yolov7-exploration/dev/v7/evaluate_quantized_model_US21_char.py", line 279, in <module>
    main()
  File "/home/econsy/projects/yolov7-exploration/lib/python3.10/site-packages/func_to_script/core.py", line 108, in scripted_function
    return func(**args)
  File "/home/econsy/projects/yolov7-exploration/dev/v7/evaluate_quantized_model_US21_char.py", line 238, in main
    torch.onnx.export(model_int8,image_tensor[None],"onnx_model.onnx",verbose=True,opset_version=15)
  File "/home/econsy/projects/yolov7-exploration/lib/python3.10/site-packages/torch/onnx/utils.py", line 506, in export
    _export(
  File "/home/econsy/projects/yolov7-exploration/lib/python3.10/site-packages/torch/onnx/utils.py", line 1548, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/home/econsy/projects/yolov7-exploration/lib/python3.10/site-packages/torch/onnx/utils.py", line 1117, in _model_to_graph
    graph = _optimize_graph(
  File "/home/econsy/projects/yolov7-exploration/lib/python3.10/site-packages/torch/onnx/utils.py", line 665, in _optimize_graph
    graph = _C._jit_pass_onnx(graph, operator_export_type)
  File "/home/econsy/projects/yolov7-exploration/lib/python3.10/site-packages/torch/onnx/utils.py", line 1891, in _run_symbolic_function
    return symbolic_fn(graph_context, *inputs, **attrs)
TypeError: quantized_add() missing 2 required positional arguments: 'op_scale' and 'op_zero_point'
4

Hi @supriyar ,
Got any leads from the above stack trace ?

Thanks

5

you can try running the model produced by convert_fx to make sure it is a valid model.

If the issue is specific to onnx lowering you can reach out to the onnx support team for more help, see Quantization — PyTorch 2.1 documentation

6

Hi @supriyar ,
Yes i could run the model produces by convert_fx and able to get the output from the model.

I have created a github issue in onnx repo. Is there any other way to reach out to the support team?

7

it could be possible that there are some float multiplications in the network, make them tensor multiplications, i.e. wrap the float value in torch.tensor()

8

hi, did you solve that problem, i got the same error with you.

9

i would look in the onnx repo as it sounds like @ramson_k created a github issue there