Quantization with custom weights in nn.Conv2d()

I want to quantize a model that I have created that uses a custom Parameter to hold the weights of several Conv2d() layers. I pass this Parameter to the forward function, which then assigns the different parts of the Parameter to the weights of the Conv2d() layers, which requires a cast to Parameter on every forward function call. This works fine for normal use, if inefficient, but when I want to use the Quantization package, the assignment throws this error:

KeyError: “attribute ‘weight’ already exists”

When I don’t use quantization, I can use the functional conv2d(), but I don’t think that is supported yet with nn.quantize. What’s the difference between a quantized model and a regular model, and how can I fix this error?


Here is a minimal example:

import torch
import torch.nn as nn
from torch.quantization import QuantStub, DeQuantStub
import copy

class Block(nn.Module):
    def __init__(self):
        super(Block, self).__init__()
        self.conv1 = nn.Conv2d(1,1,3, padding=1, bias=False)
    def forward(self, x, conv_parameter):
        self.conv1.weight = torch.nn.parameter.Parameter(conv_parameter) 
        return self.conv1(x)
class BigBlock(nn.Module):
    def __init__(self):
        super(BigBlock, self).__init__()
        self.conv_parameter = torch.nn.parameter.Parameter(torch.rand((1,1,3,3)))
        self.block = Block()
        self.quant = QuantStub()
        self.dequant = DeQuantStub()
    def forward(self, x):
        x = self.quant(x)
        x = self.block(x, self.conv_parameter)
        x = self.dequant(x)
        return x

x = torch.rand((1,1,5,5))
net = BigBlock()
print(net(x)) # works fine

qnet = copy.deepcopy(net)
qnet = qnet.eval()
qnet.qconfig = torch.quantization.default_qconfig
torch.quantization.prepare(qnet, inplace=True)
torch.quantization.convert(qnet, inplace=True)
print(qnet(x)) # throws an error

I am using pytorch-1.6.0.dev202, Python 3.8.1.

1 Like

So the issue is that I was passing floatTensor to a now torch.qint8, and attempting to cast it to a Parameter. The solution is to put an if statement like so:

class Block(torch.nn.Module):
    def __init__(self):
        super(Block, self).__init__()
        self.conv1 = torch.nn.Conv2d(1,1,3, padding=1, bias=False)
    def forward(self, x, conv_parameter):
        if(isinstance(self.conv1, torch.nn.quantized.Conv2d)):
            return self.conv1(x)
            self.conv1.weight = torch.nn.parameter.Parameter(conv_parameter) 
        return self.conv1(x)

which works since my passed in conv_parameter should be equal to the weight stored in self.conv1, so we can just ignore the conversion to Parameter when using the quantized network.

Does this code reduce the model size? I implement your code on my model and it doesn’t change my model size.