How to correctly access attribute of a custom layer inherited nn.Sequential class?

torch.package / torch::deploy
1

Hi,

I’m creating a custom layer containing 3 sub-layers which are also customized, as follows:

class CPDLayer(nn.Sequential):
     def __init__(self, in_channels, out_channels):
        super(CPDLayer, self).__init__()
        self.in_channels = in_channels
        self.out_channels = out_channels
        self.head = CPDHead(in_channels, out_channels)
        self.body = CPDBody(in_channels, out_channels)
        self.tail = CPDTail(in_channels, out_channels)
        super(CPDLayer, self).__init__(self.head, self.body, self.tail)

Now I want to access the head attribute of an object instantiated by the CPDLayer as follows:

cpd_layer = CPDLayer(Cin, Cout)
print(cpd_layer)
print(cpd_layer.head)
cpd_layer.head.weight.set_(head_factors)

But it yields an error like this:

CPDLayer(
  (0): CPDHead(in_channels=3, out_channels=64, rank=3, padding=1)
  (1): CPDBody(in_channels=3, out_channels=64, rank=3, kernel_size=3, padding=1)
  (2): CPDTail(in_channels=3, out_channels=64, rank=3, kernel_size=3, padding=1)
)
Traceback (most recent call last):
  File "/home/tien/TENDING/vgg16bn_cifar10.py", line 260, in <module>
    main()
  File "/home/tien/TENDING/vgg16bn_cifar10.py", line 109, in main
    cpd_layer = cp_decomposition_conv_layer(module, args.rank)
  File "/home/tien/TENDING/decomposition/decomposition.py", line 62, in cp_decomposition_conv_layer
    print(cpd_layer.head)
  File "/home/tien/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1614, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'CPDLayer' object has no attribute 'head'

So my questions are:

  1. I thought that with self.head in the init method, head should be an attribute of the class. Is this true? If yes, what’s wrong with my code?
  2. A not-very-related question is that my main objective is to create a wrapper class CPDLayer to wrap 3 sub-layers, so I made it with nn.Sequential as in the code above, with 2 times using super(). Is this a good practice? Or should I use self.add_module('name', object)?

Many thanks in advance!

2

[For future reference] I managed to access to attribute by modifying the class to this:

    def __init__(self, in_channels, out_channels):
        super(CPDLayer, self).__init__()
        self.in_channels = in_channels
        self.out_channels = out_channels
        self.add_module("head", CPDHead(in_channels, out_channels))
        self.add_module("body", CPDBody(in_channels, out_channels))
        self.add_module("tail", CPDTail(in_channels, out_channels))

And now the print(cpd_layer.head) works although the editor (in my case, Visual Code) doesn’t realize the head attribute (e.g, unable to point-click to leads to the parent class, sorry for my bad English).