Add layer to Faster RCNN

Hello, I am trying to add an extra layer to the backbone of the Faster RCNN model (ResNet 50) but I am having trouble with the tensor shapes if anyone can help me. The code is as below, and I am trying to determine the meta_array_shape and backbone_output_shape variables. The first one is regarding the 2nd input which is a tensor of 3 columns, while the second is trying to match the output of the backbone so that they can be concatenated together.

        meta_array_shape = 3 
        backbone_output_shape = (512,3)
        
        self.meta_model = nn.Sequential(
            nn.Linear(meta_array_shape, backbone_output_shape),
            nn.BatchNorm1d(backbone_output_shape),
            nn.ReLU(),
            nn.Dropout(p=0.5)
        )
        X = self.faster_rcnn.backbone(X)

        # Metadata Net
        Y = self.meta_model(meta_in)
        
        # Concat Feature Maps
        X = torch.cat(X, Y)

What is meta_in? What shape does it have?

The backbone_output_shape = (512,3) determines 512 as batch_size and 3 as output.
The meta_in should be (512, ?), so batch_size should be the same and then:

nn.Linear(?, 3),
nn.BatchNorm1d(3),

Where ? is any number.

Sorry, meta_in is the 2nd input tensor I mentioned above which has 3 columns. The backbone_output_shape is the original output of the faster rcnn model backbone which I am not sure how to obtain

import torch.nn

meta_array_shape = 3 
backbone_output_shape = (512,3)

meta_model = nn.Sequential(
    nn.Linear(meta_array_shape, backbone_output_shape[1]),
    nn.BatchNorm1d(backbone_output_shape[1]),
    nn.ReLU(),
    nn.Dropout(p=0.5)
)

X = torch.rand(backbone_output_shape)
Y = meta_model(torch.rand(backbone_output_shape[0], meta_array_shape))
print(torch.cat((X, Y), dim=1).shape)

or

import torch.nn

meta_array_shape = 3 
backbone_output_shape = (512,3)

meta_model = nn.Sequential(
    nn.Linear(meta_array_shape, backbone_output_shape[1]),
    nn.BatchNorm1d(backbone_output_shape[1]),
    nn.ReLU(),
    nn.Dropout(p=0.5)
)

X = torch.rand(backbone_output_shape)

Y = torch.rand(meta_array_shape)
Y = Y.repeat((backbone_output_shape[0], 1))
Y = meta_model(Y)

print(torch.cat((X, Y), dim=1).shape)

If these solutions are not good for you, then please describe the problem in more detail.

The second part is the forward function

    def forward(self, images_in, meta_in):
        X = self.faster_rcnn.transform(images_in)
        X = self.faster_rcnn.backbone(X)
        Y = self.meta_model(meta_in)
        
        X = torch.cat(X, Y)
        
        ....
                
        return X