Quantization with custom weights in nn.Conv2d()

quantization
1

I want to quantize a model that I have created that uses a custom Parameter to hold the weights of several Conv2d() layers. I pass this Parameter to the forward function, which then assigns the different parts of the Parameter to the weights of the Conv2d() layers, which requires a cast to Parameter on every forward function call. This works fine for normal use, if inefficient, but when I want to use the Quantization package, the assignment throws this error:

KeyError: “attribute ‘weight’ already exists”

When I don’t use quantization, I can use the functional conv2d(), but I don’t think that is supported yet with nn.quantize. What’s the difference between a quantized model and a regular model, and how can I fix this error?

Thanks

2

Here is a minimal example:


import torch
import torch.nn as nn
from torch.quantization import QuantStub, DeQuantStub
import copy

class Block(nn.Module):
    def __init__(self):
        super(Block, self).__init__()
        self.conv1 = nn.Conv2d(1,1,3, padding=1, bias=False)
    def forward(self, x, conv_parameter):
        self.conv1.weight = torch.nn.parameter.Parameter(conv_parameter) 
        return self.conv1(x)
    
class BigBlock(nn.Module):
    def __init__(self):
        super(BigBlock, self).__init__()
        self.conv_parameter = torch.nn.parameter.Parameter(torch.rand((1,1,3,3)))
        self.block = Block()
        self.quant = QuantStub()
        self.dequant = DeQuantStub()
    def forward(self, x):
        x = self.quant(x)
        x = self.block(x, self.conv_parameter)
        x = self.dequant(x)
        return x

x = torch.rand((1,1,5,5))
net = BigBlock()
print(net(x)) # works fine

qnet = copy.deepcopy(net)
qnet = qnet.eval()
qnet.qconfig = torch.quantization.default_qconfig
torch.quantization.prepare(qnet, inplace=True)
net(torch.rand((1,1,5,5)))
torch.quantization.convert(qnet, inplace=True)
print(qnet)
print(qnet(x)) # throws an error

I am using pytorch-1.6.0.dev202, Python 3.8.1.

1 Like
3

So the issue is that I was passing floatTensor to a now torch.qint8, and attempting to cast it to a Parameter. The solution is to put an if statement like so:

class Block(torch.nn.Module):
    def __init__(self):
        super(Block, self).__init__()
        self.conv1 = torch.nn.Conv2d(1,1,3, padding=1, bias=False)
    def forward(self, x, conv_parameter):
        if(isinstance(self.conv1, torch.nn.quantized.Conv2d)):
            return self.conv1(x)
        else:
            self.conv1.weight = torch.nn.parameter.Parameter(conv_parameter) 
        return self.conv1(x)

which works since my passed in conv_parameter should be equal to the weight stored in self.conv1, so we can just ignore the conversion to Parameter when using the quantized network.

4

Does this code reduce the model size? I implement your code on my model and it doesn’t change my model size.