Add layers on pretrained model

vision
1

I would like to fine-tune by adding layers to the resnet50 pre-trained model.
here’s resnet50 imported

from torchvision import models

resnet50 = models.resnet50(pretrained = True)
resnet50.fc = nn.Identity()

sample = torch.randn(1, 3, 224, 224)
resnet50(sample).size()

torch.Size([1, 2048])

Here are the layers to add.

class net(nn.Module):
    def __init__(self):
        super(net, self).__init__()
        self.fc = nn.Linear(2048, 128)
        
        self.branch_a1 = nn.Linear(128, 32)
        self.branch_a2 = nn.Linear(32, 1)
        
        self.branch_b1 = nn.Linear(128, 5)
    
    def forward(self, x):
        x = F.leaky_relu(self.fc(x))
        
        # branch a
        a = F.leaky_relu(self.branch_a1(x))
        out1 = self.branch_a2(a)
        
        # branch b
        out2 = self.branch_b1(x)
        
        return out1, out2

And I tie the two models together with nn.sequential.

model = nn.Sequential(resnet50, net)
model

I thought it would work, but I get an error. What should I do?

 ---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-55-cdc7b18bb3fc> in <module>
----> 1 model = nn.Sequential(resnet50, net)
      2 model

c:\users\kimsunghun\anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\container.py in __init__(self, *args)
     52         else:
     53             for idx, module in enumerate(args):
---> 54                 self.add_module(str(idx), module)
     55 
     56     def _get_item_by_idx(self, iterator, idx):

c:\users\kimsunghun\anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py in add_module(self, name, module)
    187         if not isinstance(module, Module) and module is not None:
    188             raise TypeError("{} is not a Module subclass".format(
--> 189                 torch.typename(module)))
    190         elif not isinstance(name, torch._six.string_classes):
    191             raise TypeError("module name should be a string. Got {}".format(

TypeError: __main__.net is not a Module subclass
2

Hi

You can add layers to the pre-trained model by replacing the FC layer if it’s not needed.

resnet50.fc = net()
3

Hi,

I think this post might help you:

Bests

4

the code has no problem, you just forget to create an instance of net class. I have tried your code, like this:

net_add=net()
model = nn.Sequential(resnet50, net_add)
model

and it’s output like this:

  (2): Bottleneck(
        (conv1): Conv2d(2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv3): Conv2d(512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (bn3): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
    )
    (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
    (fc): Identity()
  )
  (1): net(
    (fc): Linear(in_features=2048, out_features=128, bias=True)
    (branch_a1): Linear(in_features=128, out_features=32, bias=True)
    (branch_a2): Linear(in_features=32, out_features=1, bias=True)
    (branch_b1): Linear(in_features=128, out_features=5, bias=True)
  )
)
5

you can reference this answer:https://stackoverflow.com/questions/44406819/pytorch-custom-layer-is-not-a-module-subclass

6

All of the above are adding the layers at the end of the pre-trained network. But how to add layers in the middle of the pretrained model?

7

when we train the new layers cannot get the weights from pretrained because these are not defined what to do ?