Removing pretrained Vgg16 4th max pool layer

vision
1

Initailly, I had the entire pretrained Vgg16 model which the last[5th max pooling layer] removed

vgg = models.vgg16()
vgg.load_state_dict({k:v for k,v in state_dict.items() if k in vgg.state_dict()})

self.RCNN_base = nn.Sequential(*list(vgg.features._modules.values())[:-1])##original

I wanted to drop the 4th max-pooling layer too:

So I changed the above code to:

   vgg = models.vgg16()
   vgg.load_state_dict({k:v for k,v in state_dict.items() if k in vgg.state_dict()})
   list1 = list(vgg.features._modules.values())[:23]
   list1.extend(list(vgg.features._modules.values())[24:-1])
   self.RCNN_base_new = nn.Sequential(*list1)

This is my model summary after removing the 4th max pool

Sequential(
  (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (1): ReLU(inplace)
  (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (3): ReLU(inplace)
  (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (6): ReLU(inplace)
  (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (8): ReLU(inplace)
  (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (11): ReLU(inplace)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (13): ReLU(inplace)
  (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (15): ReLU(inplace)
  (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (18): ReLU(inplace)
  (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (20): ReLU(inplace)
  (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (22): ReLU(inplace)
  (23): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (24): ReLU(inplace)
  (25): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (26): ReLU(inplace)
  (27): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (28): ReLU(inplace)
)

the code is running but now the accuracy of prediction is zero
Have I removed the pooling layer correctly?

1 Like
2

Are you using torchvision for the base model? If so, have you tried passing in the pretrained=True flag?

3

So I modifies my code a bit to the following. Still it doesn’t seem to work

    pretrained = models.vgg16(pretrained=True)
   
    vgg = my_model()# vgg16 with 4th pool removed
    vgg_dict = vgg.state_dict()

    for k, v in pretrained_dict.items():
        if int(k.split('.')[1]) > 22:
            break

        vgg_dict[k] = v
     

    vgg_dict['features.23.weight'] = pretrained_dict['features.24.weight']
    vgg_dict['features.23.bias'] = pretrained_dict['features.24.bias']
    vgg_dict['features.25.weight'] = pretrained_dict['features.26.weight']
    vgg_dict['features.25.bias'] = pretrained_dict['features.26.bias']
    vgg_dict['features.27.weight'] = pretrained_dict['features.28.weight']
    vgg_dict['features.27.bias'] = pretrained_dict['features.28.bias']