Good advice to double the input and output channel from the pre-trained model?

complex
1

Hi,

I have loaded the pre-trained ResNet-50 model, and now I want to change the input and output channels of all layers. Not only the first or last layers. But maintaining the same model structure.

is there any way to do this?

Here is a simple example of what I want to do.

class CustomCNN(nn.Module):
    def __init__(self):
        super(MiniCustomCNN, self).__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 6, (5, 5)),
            nn.ReLU(),
            nn.MaxPool2d((2, 2)),
            nn.Conv2d(6, 8, (5, 5)),
            nn.ReLU(),
            nn.MaxPool2d((2, 2))
        )
        self.fc = nn.Sequential(
            nn.Linear(200, 40),
            nn.Linear(40, 10)
        )

    def forward(self, x):
        x = self.features(x)
        x = torch.flatten(x, 1)
        x = self.fc(x)
        return x

model = CustomCNN()
# To do something...

The model summary results I am aiming for…

CustomCNN(
(features): Sequential(
(0): Conv2d(6, 12, kernel_size=(5, 5), stride=(1, 1))
(1): ReLU()
(2): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), padding=0, dilation=1, ceil_mode=False)
(3): Conv2d(12, 16, kernel_size=(5, 5), stride=(1, 1))
(4): ReLU()
(5): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), padding=0, dilation=1, ceil_mode=False)
)
(fc): Sequential(
(0): Linear(in_features=400, out_features=80, bias=True)
(1): Linear(in_features=80, out_features=10, bias=True)
)
)

Simply, I want to double(or customize) the input and output channels from the pre-build model.

Any help will be appreciated.

Thanks.

2

Not a professional here but here is how I’d approach the situation.

Here is what I understood from your question, you have downloaded a pre-trained model, and now you want to change the internal channels of the layers, but keep the layers themselves, right?

Would you mind explaining further why you would need to do this? Maybe it would shed some light on how best to solve the issue.

I’m not sure if you can change the channels without hurting the weights because you’d have to reinitialize the layers to get the channels you want.

A solution might be to use the pre-trained model as an encoder, pass your data through the encoder, then create a custom network and pass it through. e.g

class CustomCNN(nn.Module):
    def __init__(self, encoder):
        super(CustomCNN, self).__init__()
        # pass the pretrained model here
        self.encoder = encoder
        # define your custom architecture
        self.features = nn.Sequential(
            nn.Conv2d(3, 6, (5, 5)),
            nn.ReLU(),
            nn.MaxPool2d((2, 2)),
            nn.Conv2d(6, 8, (5, 5)),
            nn.ReLU(),
            nn.MaxPool2d((2, 2))
        )
        self.fc = nn.Sequential(
            nn.Linear(200, 40),
            nn.Linear(40, 10)
        )

    def forward(self, x):
       # pass through encoder first
        x = self.encoder(x)
        x = self.features(x)
        x = torch.flatten(x, 1)
        x = self.fc(x)
        return x

model = CustomCNN()

This way, your model works on whatever channels you have.

I’m not sure this is possible because you have to redefine

The tricky part would be how to match the output shape of the encoder to your first Conv2D.

1 Like
3

Thanks for your reply.

I am implementing channel-growing models for my business purpose.

Training from the small size of data and minimal model, and then upsize the model and data size after certain epochs.

Previous many upsizing model methods have been implemented by adding layers, but for me, I am trying to upsize the model by channels. (Migrate the small size of model’s weights into some part of the big model and retrain with new data)

Therefore, It is ok with hurting the weights of the pre-trained model.

Maybe I can make a big-size model by hard coding, but I want to make a kind of automated growing model.

By the way, your suggestion also makes sense to me :slight_smile:

I will try if it works as I intended.

Thanks.