AttributeError: 'Conv2d' object has no attribute '_modules'

1

Trying to save entire model like in this docs Save/Load Entire Model and then load it, but got this error AttributeError: 'Conv2d' object has no attribute '_modules'.

I saved model like this (I post only part of model, becaus it’s to big):

torch.save(model_static_quantized, "full_model.pth")

And load model like this:

model_filename = "D:/3divi_cv/hw_9/model/full_model.pth"

checkpoint = torch.load(model_filename) #, map_location=torch.device('cpu')))
checkpoint.eval()

What is important is that I used quantization before saving the model and my deeplabv3_resnet50 model starts look like this:

QuantDeQuantNet(
(quant): Quantize(scale=tensor([0.0079]), zero_point=tensor([0]), dtype=torch.quint8)
(dequant): DeQuantize()
(model): DeepLabV3(
(backbone): IntermediateLayerGetter(
(conv1): QuantizedConv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), scale=0.07399188727140427, zero_point=53, padding=(3, 3), bias=False)
(bn1): QuantizedBatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(layer1): Sequential(
(0): Bottleneck(
(conv1): QuantizedConv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), scale=0.04221147298812866, zero_point=63, bias=False)
(bn1): QuantizedBatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): QuantizedConv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), scale=0.03571641072630882, zero_point=62, padding=(1, 1), bias=False)
(bn2): QuantizedBatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv3): QuantizedConv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.007678185123950243, zero_point=68, bias=False)
(bn3): QuantizedBatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)

How can I fix given error?

2

Storing the model directly can easily break as you would need to restore the same data/file structure of your project and we thus recommend storing the state_dict instead and to recreate the model.
Did you try to use this approach and would it work?

3

I tried to save state_dict, but I don’t understande, how can I load it as model with architecture. I guess that state_dict save only weights. I load the model in this order:

  1. model = deeplabv3_resnet50()
  2. model.load_state_dict(‘my_saved_model_dict’)

And get this error:

1600 0, 'Missing key(s) in state_dict: {}. '.format(
1601 ‘, ‘.join(’“{}”’.format(k) for k in missing_keys)))
1603 if len(error_msgs) > 0:
→ 1604 raise RuntimeError(‘Error(s) in loading state_dict for {}:\n\t{}’.format(
1605 self.class.name, “\n\t”.join(error_msgs)))
1606 return _IncompatibleKeys(missing_keys, unexpected_keys)

RuntimeError: Error(s) in loading state_dict for DeepLabV3:
Unexpected key(s) in state_dict: “backbone.conv1.bias”, “backbone.conv1.scale”, “backbone.conv1.zero_point”, “backbone.bn1.scale”, “backbone.bn1.zero_point”, “backbone.layer1.0.conv1.bias”, “backbone.layer1.0.conv1.scale”, “backbone.layer1.0.conv1.zero_point”, “backbone.layer1.0.bn1.scale”, “backbone.layer1.0.bn1.zero_point”, “backbone.layer1.0.conv2.bias”, “backbone.layer1.0.conv2.scale”, “backbone.layer1.0.conv2.zero_point”, “backbone.layer1.0.bn2.scale”, “backbone.layer1.0.bn2.zero_point”, “backbone.layer1.0.conv3.bias”, “backbone.layer1.0.conv3.scale”, “backbone.layer1.0.conv3.zero_point”, “backbone.layer1.0.bn3.scale”, “backbone.layer1.0.bn3.zero_point”

After the model is quantized, its modules become different from the modules of the original model and it is not possible to load the weights

How can I save ALL model with architecture and weights, to be able to simply load it all and simply assign it to a model variable like this:

model = torch.load(‘path_to_saved_model’)

Also thank you for your help. I realy appreciate it

4

You should create the model in the same way, i.e. also with the quantization, before trying to load the state_dict.

Again, I would not recommend it but using a plain torch.save(model, PATH) should work assuming you are restoring all files and definitions in the same way.

5

I’m trying load my model like you said, but this time I have an error when I’m trying to use this model to predict. I load model like in this code:

class QuantDeQuantNet(torch.nn.Module):
    def __init__(self, model):
        super(QuantDeQuantNet, self).__init__()
        self.quant = torch.quantization.QuantStub()
        self.dequant = torch.quantization.DeQuantStub()
        self.model = model

    def forward(self, x):
        x = self.quant(x)
        x = self.model(x)
        x = self.dequant(x)
        return x


class Solver:
    def __init__(self, weights_filename):
        self.model = self.init_model(weights_filename)
        self.device = 'cpu'


    def createDeepLabv3(self, outputchannels=2):
        model = models.segmentation.deeplabv3_resnet50(pretrained=True, progress=True, aux_loss=True)
        model.classifier = DeepLabHead(2048, outputchannels)

        model.aux_classifier[-1] = nn.Conv2d(256, 2, kernel_size=(1, 1), stride=(1, 1))
        model_fp32 = QuantDeQuantNet(model).to('cpu')

        backend = "fbgemm"
        model_fp32.qconfig = torch.quantization.get_default_qconfig(backend)
        torch.backends.quantized.engine = backend

        model_static_quantized = torch.quantization.prepare(model_fp32, inplace=False)
        model_static_quantized = torch.quantization.convert(model_static_quantized,
                                                            inplace=False)
        

        return model_static_quantized


    def init_model(self, weights_filename):
        model = self.createDeepLabv3()

        if torch.cuda.is_available():
            checkpoint = (torch.load(weights_filename))
        else:
            checkpoint = (torch.load(weights_filename, map_location=torch.device('cpu')))

        model.model.load_state_dict(checkpoint)
        print(model)
        return model

And get this huge error

NotImplementedError: Could not run ‘aten::add.out’ with arguments from the ‘QuantizedCPU’ backend. This could be because the operator doesn’t exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit Internal Login for possible resolutions. ‘aten::add.out’ is only available for these backends: [Dense, FPGA, Conjugate, VmapMode, FuncTorchGradWrapper, Functionalize, MPS, IPU, UNKNOWN_TENSOR_TYPE_ID, QuantizedXPU, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseCPU, SparseCUDA, SparseHIP, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseXPU, UNKNOWN_TENSOR_TYPE_ID, SparseVE, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, NestedTensorCUDA, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID].

CPU: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterCPU.cpp:37386 [kernel]
CUDA: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterCUDA.cpp:51977 [kernel]
Meta: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterMeta.cpp:31637 [kernel]
MkldnnCPU: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterMkldnnCPU.cpp:690 [kernel]
SparseCPU: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterSparseCPU.cpp:1858 [kernel]
SparseCUDA: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterSparseCUDA.cpp:2018 [kernel]
SparseCsrCPU: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterSparseCsrCPU.cpp:1507 [kernel]
SparseCsrCUDA: registered at C:\cb\pytorch_1000000000000\work\build\aten\src\ATen\RegisterSparseCsrCUDA.cpp:1657 [kernel]
BackendSelect: fallthrough registered at C:\cb\pytorch_1000000000000\work\aten\src\ATen\core\BackendSelectFallbackKernel.cpp:3 [backend fallback]
Python: registered at C:\cb\pytorch_1000000000000\work\aten\src\ATen\core\PythonFallbackKernel.cpp:133 [backend fallback]
Named: fallthrough registered at C:\cb\pytorch_1000000000000\work\aten\src\ATen\core\NamedRegistrations.cpp:11 [kernel]
Conjugate: registered at C:\cb\pytorch_1000000000000\work\aten\src\ATen\ConjugateFallback.cpp:18 [backend fallback]
Negative: registered at C:\cb\pytorch_1000000000000\work\aten\src\ATen\native\NegateFallback.cpp:18 [backend fallback]
ZeroTensor: registered at C:\cb\pytorch_1000000000000\work\aten\src\ATen\ZeroTensorFallback.cpp:86 [backend fallback]

I tried to run the model using both cpu and cuda. Nothing helps.

I also noticed that when trying to quantize my other model, this two blocks

(quant_m): Quantize(scale=tensor([0.0157]), zero_point=tensor([64]), dtype=torch.quint8)
(dequant_m): DeQuantize()

placed in the end of model:

QuantizedSiameseNetwork(
(backbone): Backbone(
(conv1): QuantizedConv2d(3, 8, kernel_size=(3, 3), stride=(1, 1), scale=0.03301658481359482, zero_point=68, padding=(1, 1))
(pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv2): QuantizedConv2d(8, 16, kernel_size=(3, 3), stride=(1, 1), scale=0.04873496666550636, zero_point=46, padding=(1, 1))
(conv3): QuantizedConv2d(16, 32, kernel_size=(3, 3), stride=(1, 1), scale=0.06440312415361404, zero_point=85, padding=(1, 1))
(conv4): QuantizedConv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), scale=0.04743369296193123, zero_point=93, padding=(1, 1))
(fc1): QuantizedLinear(in_features=3136, out_features=1024, scale=0.015503729693591595, zero_point=72, qscheme=torch.per_channel_affine)
(fc2): QuantizedLinear(in_features=1024, out_features=2, scale=0.024672511965036392, zero_point=80, qscheme=torch.per_channel_affine)
)
(quant_m): Quantize(scale=tensor([0.0157]), zero_point=tensor([64]), dtype=torch.quint8)
(dequant_m): DeQuantize()
)

But in my current model that I’m trying quantize thit to blocks place at the beginning of model:

QuantDeQuantNet(
(quant): Quantize(scale=tensor([0.0079]), zero_point=tensor([0]), dtype=torch.quint8)
(dequant): DeQuantize()
(model): DeepLabV3(
(backbone): IntermediateLayerGetter(