How to modify the first and last layer of pretrained network

Hello. I have VGG19 network and i want modify the the first and last layer.

I want the modified network have 6 channel input channel. and the weight for first three channel is that of VGG 19.

In addition, I want to modify the last layer of VGG (FC layer) with output channel 1

How can i do this?

You can write your code for the modified VGG network and then manually load weights into each layer.

hello, did you solve that issu?i have the same experience as you.

class VGG(nn.Module):

    def __init__(self, features, num_classes=1000, init_weights=True):
        super(VGG, self).__init__()
        self.features = features
        self.classifier = nn.Sequential(
            nn.Linear(512 * 7 * 7, 4096),
            nn.ReLU(True),
            nn.Dropout(),
            nn.Linear(4096, 4096),
            nn.ReLU(True),
            nn.Dropout(),
            nn.Linear(4096, num_classes),
        )
        if init_weights:
            self._initialize_weights()

    def forward(self, x):
        x = self.features(x)
        x = x.view(x.size(0), -1)
        x = self.classifier(x)
        return x
    def _initialize_weights(self):
        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
                if m.bias is not None:
                    nn.init.constant_(m.bias, 0)
            elif isinstance(m, nn.BatchNorm2d):
                nn.init.constant_(m.weight, 1)
                nn.init.constant_(m.bias, 0)
            elif isinstance(m, nn.Linear):
                nn.init.normal_(m.weight, 0, 0.01)
                nn.init.constant_(m.bias, 0)

cfg = {
    'A': [64, 'M', 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'],
    'B': [64, 64, 'M', 128, 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M'],
    'D': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 'M', 512, 512, 512, 'M', 512, 512, 512, 'M'],
    'E': [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 256, 'M', 512, 512, 512, 512, 'M', 512, 512, 512, 512, 'M'],
}
def vgg19(pretrained=False, **kwargs):
    """VGG 19-layer model (configuration "E")
    Args:
        pretrained (bool): If True, returns a model pre-trained on ImageNet
    """
    if pretrained:
        kwargs['init_weights'] = False
    model = VGG(make_layers(cfg['E']), **kwargs)
    if pretrained:
        model.load_state_dict(model_zoo.load_url(model_urls['vgg19']))
    return model
def make_layers(cfg, batch_norm=False):
    layers = []
    in_channels = 6 #Define the number of channels
    for v in cfg:
        if v == 'M':
            layers += [nn.MaxPool2d(kernel_size=2, stride=2)]
        else:
            conv2d = nn.Conv2d(in_channels, v, kernel_size=3, padding=1)
            if batch_norm:
                layers += [conv2d, nn.BatchNorm2d(v), nn.ReLU(inplace=True)]
            else:
                layers += [conv2d, nn.ReLU(inplace=True)]
            in_channels = v
    return nn.Sequential(*layers)
vgg19(pretrained=False)

The Above is the VGG19 with 6 input channels.
For the weight you need to load it manually for every layer through iterating.

One solution would be us vgg19(pretrained=True) with input channels=3 then you will get net weights as ‘vgg19-dcbb9e9d.pth’ file then see my other reply

use flat_weights,shapes=flattenNetwork(vgg19_3channels)
x=unFlattenNetwork(flat_weights,shapes) --this will give you the numpy array for each layer.

then you modify the first x[0] which is 3 channel in the above to the 6 channels just by adding your weights or dummy weights.
then iterate through all layers and bias and modify the vgg19_6channel model with modified x you got above.

 vgg19_6channel.conv1.weight.data = torch.from_numpy(x[0])
 vgg19_6channel.conv1.bias.data =   torch.from_numpy(x[1]) ..etc
``
1 Like

This worked for me: I was trying to use greyscale image.

vgg16 = models.vgg16_bn(pretrained=True)
for param in vgg16.parameters():
param.requires_grad = False
layers = list(vgg16.features.children())[:-1]
layers[0] = nn.Conv2d(1, 64, kernel_size=3, stride=1, padding=1)
features = nn.Sequential(*layers).cuda()

Hi @Divya_B could explain how it is modified in the code snippet which you have written? I am trying to change the number of input channels to 4 in vgg19. If you could explain a little it would help me with my task.