RuntimeError: No function is registered for schema aten::thnn_conv2d_forward(Tensor self, Tensor weight, int[2] kernel_size, Tensor?

After static . quantization, I receive the following error-

RuntimeError: No function is registered for schema aten::thnn_conv2d_forward(Tensor self, Tensor weight, int[2] kernel_size, Tensor? bias, int[2] stride, int[2] padding) -> (Tensor output, Tensor finput, Tensor fgrad_input) on tensor type QuantizedCPUTensorId; available functions are CPUTensorId, VariableTensorId

raised by traced_script_module = torch.jit.trace(quantized, q_example, check_trace=False)

full code here - https://github.com/raghavgurbaxani/experiments/blob/master/try_static_quant.py

This is saying conv2d_forward is given a quantized tensor, which means some float Conv is not swapped as quantized Conv, can you print the quantized model and see where this happens?

Hello,

I am not sure this is the same problem or not.
But for my example, when I forget to add “model.eval()”, there is such error.
When I add “model.eval()”, it seems fine.

I only get this error when I load the model with GPU:

use_cuda = torch.cuda.is_available()
device = torch.device("cuda:0" if use_cuda else "cpu")

model = ResNet3D().to(device)
model.apply(weights_init)

input = torch.rand([2,1,86,110,78])
output = model(input) 
print(input.shape)
print(output.shape)

Then comes the error: RuntimeError: No function is registered for schema aten::thnn_conv3d_forward(Tensor self, Tensor weight, int[3] kernel_size, Tensor? bias, int[3] stride, int[3] padding) -> (Tensor output, Tensor finput, Tensor fgrad_input) on tensor type CUDATensorId; available functions are CPUTensorId, VariableTensorId

if I load the model in cpu:

model = ResNet3D()
model.apply(weights_init)
output = model(input)
print(input.shape)
print(output.shape)

gives output nicely:

torch.Size([2, 1, 86, 110, 78])
torch.Size([2, 256])

solution: I will correct myself. I loaded the input as following:

input = torch.rand([2,1,86,110,78]).float().to(device)

which solved the issue of loading model with GPUs.
Thanks

2 Likes