Supported quantized tensor operations

Is there a list of currently supported operations for quantized tensors?

I run into issues quantizing a network requiring tensor additions:

RuntimeError: Could not run 'aten::add.Tensor' with arguments from the 'QuantizedCPUTensorId' backend. 'aten::add.Tensor' is only available for these backends: [SparseCPUTensorId, CPUTensorId, VariableTensorId, MkldnnCPUTensorId].

Others have reported running into issues with div and cat operations - I presume these are also not supported atm.

In case this is helpful to anyone, there are:

https://pytorch.org/docs/stable/quantization.html#torch.nn.quantized.FloatFunctional

and

https://pytorch.org/docs/stable/quantization.html#torch.nn.quantized.QFunctional

that support add and other operations.

2 Likes

Add is supported, but not as a at::add.Tensor. The reason is that addition (or any arithmetic) requires output scale/zero_point. Also, often times quantized ops need to be stateful. Hence, there are stateful FloatFunctional and QFunctional

1 Like