RuntimeError: Could not run 'aten::thnn_conv2d_forward' with arguments from the 'QuantizedCPUTensorId' backend. 'aten::thnn_conv2d_forward' is only available for these backends: [CPUTensorId, VariableTensorId]

quantization
21

It turned out this is not suppoted by pytroch quantization

22

this is typically an error resulted from misplacement of QuantStub/DeQuantStub in eager mode, did you place these things correctly for conv? Assuming Conv2dSameExport is a custom module that uses conv, if you wan to quantize the conv inside this module, you’ll need to make sure place QuantStub and DeQuantStub around the conv module.

23

hi @jerryzh168

Could not run ‘aten::thnn_conv_depthwise2d_forward’ with arguments from the ‘QuantizedCUDA’ backend…

I am getting this as well.

(1): Sequential(
(0): BaseConv(
(conv): QuantizedConv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), scale=0.10585608333349228, zero_point=146, padding=(1, 1))
(bn): Identity()
(act): SiLU(inplace=True)
)
(1): BaseConv(
(conv): QuantizedConv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), scale=0.14361223578453064, zero_point=136, padding=(1, 1))
(bn): Identity()
(act): SiLU(inplace=True)
)
)
(2): Sequential(
(0): BaseConv(
(conv): QuantizedConv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), scale=0.08742094039916992, zero_point=134, padding=(1, 1))
(bn): Identity()
(act): SiLU(inplace=True)
)
(1): BaseConv(
(conv): QuantizedConv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), scale=0.10120078176259995, zero_point=141, padding=(1, 1))
(bn): Identity()
(act): SiLU(inplace=True)
)
)
…
I am using quant → model → dequant. Does this mean I have to go into the model and add quant and dequant to every single conv2d i see?

24

looks like you have QuantizedConv2d module in model, not sure how you get ‘aten::thnn_conv_depthwise2d_forward’ , are you using functional conv in your code?

25

ah yes, i have changed it to nn.Conv2d instead. Just to confirm, functional vs nn have the same function and its better to use nn.Conv2d in quantization?

26

yeah function and nn module is using the same functions. but eager mode quantization only supports quantization in the level of module, and there is no support for functionals at all.

27

if you want to quantize functionals, please feel free to try out our fx graph mode quantization tool: (prototype) FX Graph Mode Post Training Static Quantization — PyTorch Tutorials 1.9.1+cu102 documentation