How to match the output of one convolutional layer with the input of the next one?

vision
1

Let’s say I have this small architecture and I want to remove conv3. However, since I have pre-trained weights, I can’t (or don’t know how) directly change the output of conv2 to match conv4. Is there a way to resize the input of the fourth sequential without doing a convolution?

        # conv2
        self.conv2 = nn.Sequential(
            nn.MaxPool2d(2, stride=2, ceil_mode=True),  # 1/2
            nn.Conv2d(64, 128, 3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(128, 128, 3, padding=1),
            nn.ReLU(inplace=True),
        )

        # conv3
        """
        self.conv3 = nn.Sequential(
            nn.MaxPool2d(2, stride=2, ceil_mode=True),  # 1/4
            nn.Conv2d(128, 256, 3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(256, 256, 3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(256, 256, 3, padding=1),
            nn.ReLU(inplace=True),
        )"""

        # conv4
        self.conv4 = nn.Sequential(
            nn.MaxPool2d(2, stride=2, ceil_mode=True),  # 1/8
            #Something here that matches 128 with 256 without performing another convolution?
            nn.Conv2d(256, 512, 3, padding=1), 
            nn.ReLU(inplace=True),
            nn.Conv2d(512, 512, 3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(512, 512, 3, padding=1),
            nn.ReLU(inplace=True),
        )
2

You could use nn.Upsample to manipulate the spatial size of your activation.
However, since your model is pretrained, you might need to finetune it again, as the input to self.conv4 contains now new activations.