Is there a way to quantize conv_transpose2d layer?

@ruka – For the SNR metric – the higher is the better and I usually go with the rule of thumb of 15-20dB is good for quantization. However, this number might not be good for all the models. I’ll take a look at the model + data you sent

@Zafar Thank you so much :smiley:

@Zafar, I am facing the same issue. It seems to be a Keypoints issue. I trained same model on Keypoints data and classification data ( just last layer changed). I got following results on quantization.

Trained for Keypoints:
Weights SNR: 40 - 47 (almost perfect)
model outputs stats SNR: 23(first layer) - 0.01 (end layers). The value kept on reducing with every layer. Why there is no correlation between weights and output.
Quantized model accuracy: 0.01

Trained for Classification:
Weights SNR: 43-48
model outputs stats SNR: 10-22 (random)
Float model accuracy: 92%
quantized model accuracy: 91.6%

I will be waiting for your analysis.

@Mamta, @ruka Thank you for reporting this – I will take a look at these models. I currently have a similar issue for the generator models – not sure if it is related, but I will be out looking at this issue more closely

hi @Zafar: I am running into this error when trying to use a ConvTranspose2d with a FBGEMM backend. I am trying to implement workaround (1) that you suggested, i.e. to wrap the ConvTranspose2d with dequant and quant steps, but am struggling to get it right. That is, even if I run those steps, the error (‘FBGEMM doesn’t support transpose packing yet!’) appears. Do you know of an example of how to implement such wrapping that would allow quantization to go through? Thank you!

What I did was to create QuantStub and DeQuantStub instances during initialization, and then during the forward() did

        x = self.dequant(x)
        x = self.transpose_conv2d(x)
        x = self.quant(x)

… but this still gives the same error.

Hi, @rfejgin , may I know which version of pytorch you are using? Did they announce that ConvTranspose2d is supported on FBGEMM backend in their latest release? As far as I know, about a month ago, ConvTranspose2d is still not supported on FBGEMM backend yet.

I’m on 1.7.0. Yes, I’m aware that ConvTranspose2d is not supported on the FBGEMM backend, so I was trying the workaround suggested by @Zafar, which is to wrap the calls to the module with dequant->conv2d_transpose->quant, but couldn’t get that to work.

Hi, @rfejgin According to official tutorial https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html#model-architecture, dequant->conv2d_transpose->quant is a standard process for quantization, it doesn’t tell pytorch which backend it should use.
Could you try setting qconfig to QNNPACK as @Zafar mentioned in step 2) ?

Okay, I think I figured out my problem. I was setting the qconfig correctly

backend = "fbgemm"                
m.qconfig = torch.quantization.get_default_qconfig(backend)

but didn’t realize that I also needed to choose the backend, like so:

torch.backends.quantized.engine = backend

With this change I am able to get through the quantization conversion (convert()) using FBGEMM and conv_transpose2d. So maybe they did add FBGEMM support after all?

Thanks @ruka for the pointers and suggestions, that tutorial was helpful.

1 Like

Hmm, actually turns out I’m still getting the error. Not sure what is making it come and go (I’ve also been playing around with the ‘inplace’ parameter), will try to narrow down.

You getting a new error or still ‘FBGEMM doesn’t support transpose packing yet’ ?

Still “FBGEMM doesn’t support transpose packing yet”.

It appears that as long as a module being quantized contains a conv_transpose2d module, quantization will fail when using the FBGEMM backend, with the error
RuntimeError: FBGEMM doesn't support transpose packing yet!.

This happens even if the forward() function never calls the conv_transpose2d! And then it’s not surprising that wrapping the transposed convolution using dequant->conv_transpose2d->quant also doesn’t work.

Here’s a fairly minimal example:

import torch
import torch.nn as nn
import torch.quantization
from torch.quantization import QuantStub, DeQuantStub

class UpSample(nn.Module):
    def __init__(self, C):
        super(UpSample, self).__init__()
        self.transpose_conv2d = nn.ConvTranspose2d(in_channels=C, out_channels=C, kernel_size = (2,2))
        self.quant = QuantStub()
        self.dequant = DeQuantStub()

    def forward(self, x):
        # Enabling the following 3 lines doesn't help. 
        #x = self.dequant(x)
        #x = self.transpose_conv2d(x)
        #x = self.quant(x)

        # Even if forward() doesn't call the transposed convolution, we get an error.
        return x

C = 10
model = UpSample(C=C)
x = torch.ones(1, C, 3, 1)

# Set up quantization
backend = 'fbgemm'
model.qconfig = torch.quantization.get_default_qconfig(backend)
torch.backends.quantized.engine = backend

# Insert observers
model = torch.quantization.prepare(model, inplace=True)
# Calibrate
_ = model(x)
# Quantize
m = torch.quantization.convert(model, inplace=True)

Result:

Exception has occurred: RuntimeError
FBGEMM doesn't support transpose packing yet!

@Zafar, @jerryzh168: do you know of any way to work around fact that conv_transpose2d cannot be quantized when using the FBGEMM backend? I don’t mind keeping that layer unquantized, but there doesn’t seem to be a way to get around the above error, even if dequantizing before the transposed conv. Thank you - any advice would be appreciated!

I think the fbgemm convtranspose is landed – can you try again?

I’m on 1.7.0, did you mean it is landed on latest master? I can try that.

Also, for anyone else trying to wrap an unsupported layer type with dequant-layer-quant, the following appears to work, at least in the sense that quantization completes without error:

class UpSample(nn.Module):
    def __init__(self, C):
        super(UpSample, self).__init__()
        self.transpose_conv2d = nn.ConvTranspose2d(in_channels=C, out_channels=C, kernel_size = (2,2))

        self.quant = QuantStub()
        self.dequant = DeQuantStub()
        # disable quantization on unsupported layers types
        self.transpose_conv2d.qconfig = None

    def forward(self, x):
        x = self.dequant(x)
        x = self.transpose_conv2d(x)
        x = self.quant(x)
        return x

I tried the latest nightly build and indeed it looks like conv_transpose2d is supported now even on FBGEMM. But I am getting this error:
Per Channel Quantization is currently disabled for transposed conv

My network is using the default qconfig for FBGEMM, and the inputs to the network are quantized with a QuantStub() at the start of the network. Is there something special I should be doing to choose a different quantization type? thank you.

This is odd – I thought I landed the change. I will take a look later this week

1 Like

@Zafar thank you for all your work!

I’ve just tried to quantize a model with ConvTranspose1d using torch 1.8 and fbgemm backend, but got this error message: “AssertionError: Per channel weight observer is not supported yet for ConvTranspose{nx}d.”

Could you maybe prompt when this observer will be released, is it in progress?

At the moment there is no active work to implement the per channel observer for the convtranspose. The reason is that there is non-trivial task that requires observation of a proper channel, which is different for the conv and convtranspose. If you add a feature request on github, I will try to get to it as soon as I can. Meanwhile, you should use per tensor configuration.

thank you for a quick response! Added an issue

1 Like