Modify Layers of Faster RCNN

vision
1

M using the object detection model Faster RCNN Resnet 50 for some project. Now I want to add some additional layers to it.
How can I add additional layers to he existing pre-trained object detection model ?

import torch
import torch.nn as nn
from torchvision.ops.misc import FrozenBatchNorm2d
from torch.nn import Conv2d, Sequential, ReLU

class CustomNet(nn.Module):
    def __init__(self):
        super(CustomNet, self).__init__()

        self.l1 = Sequential(
            Conv2d(2048, 512, kernel_size=(1,1), bias = False, stride = (1,1)),
            FrozenBatchNorm2d(512),
            Conv2d(512, 512, kernel_size=(3,3), bias = False, stride = (1,1), padding=(1, 1)),
            FrozenBatchNorm2d(512),
            Conv2d(512, 2048, kernel_size=(1,1), bias = False, stride = (1,1)),
            FrozenBatchNorm2d(2048),
            ReLU(inplace=True) )

    def forward(self, x):
        x = self.l1(x)
        return x

I want to add the above layer generated to the main model.

main_model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)

How can I do that ?

2

You could do this: Add a the model in init like this

self.main_model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)

and then in the forward pass you could do this

x = self.main_model(x)
x = self.l1(x)
3

Thank you for the response @Dwight_Foster .
I did that, now I realized that my whole question was wrong. I want to insert the layer in between the existing Sequential layers of FRCNN model.
As FRCNN model have on 4 elements Rransform, Backbone, RPN and ROI and I want to add layers in backbone of the network.

4

Ok for the backbone you could add a layer by using the .add_module function. For example if you wanted to add a convolution to layer 3 you could do this

main_model.backbone.body.layer3.add_module("6", nn.Sequential(nn.Conv2d(4,4,5,6)))
5

@Dwight_Foster I was able to add the layers as you mentioned an it works like a charm. But, I have another question what is Bottleneck in the code below:

main_model.backbone.body.layer1.add_module("3", nn.Sequential(nn.Conv2d(1024,256,kernel_size=(1,1),stride=(1,1),bias=False)))

After adding the layer an executing the code below

main_model.backbone.body.layer1

I get this output

Sequential(
  (0): Bottleneck(
    (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn1): FrozenBatchNorm2d(64)
    (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    (bn2): FrozenBatchNorm2d(64)
    (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn3): FrozenBatchNorm2d(256)
    (relu): ReLU(inplace=True)
    (downsample): Sequential(
      (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (1): FrozenBatchNorm2d(256)
    )
  )
  (1): Bottleneck(
    (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn1): FrozenBatchNorm2d(64)
    (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    (bn2): FrozenBatchNorm2d(64)
    (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn3): FrozenBatchNorm2d(256)
    (relu): ReLU(inplace=True)
  )
  (2): Bottleneck(
    (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn1): FrozenBatchNorm2d(64)
    (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
    (bn2): FrozenBatchNorm2d(64)
    (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
    (bn3): FrozenBatchNorm2d(256)
    (relu): ReLU(inplace=True)
  )
  (3): Sequential(
    (0): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
  )
)

As you can see it is already a sequential layer and I dont know what is Bottleneck mean ?

6

Glad that it worked. The Bottleneck is just a technique in resnet that they used to lower the number of parameters. A complete explanation can be found here.

7

hi @draaken0 i am trying to add a classification layer in between the pretrained model Faster RCNN Resnet 50. as i am not so expert in PyTorch. i cant able to find how to add the layer! can you able to help me?

8

Hi @Dwight_Foster I am trying to add a Block of layer to Faster RCNN Resnet 50 pretrained model as the model is giving the output of prediction box and the object class, i want to add a parallel branch which consist of sequential layer followed by average pooling,convolution,average pooling and fully connected layer followed by soft max function to identify the object class is affected or not.

Hence the output of the pretrained model can give the object class,object coordinates and affected or not (classify) after adding the new layer.

9

do you get the solution for that? i have simmilar case with you when i want to add some layer for other classification task.