How to replace last layer in Sequential

1

I have a pretrained model with layers stacked in nn.Sequential (I’m using ResNext from link)
And I want to replace only the last Linear layer.

I first load the pretrained model and weights as below,

model = resnext_101_64x4d.resnext_101_64x4d
model.load_state_dict(torch.load('resnext_101_64x4d.pth'))

Then how can I replace the last layer?
the model looks like below,

Sequential(
  (0): Conv2d (3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True)
  (2): ReLU()
   ...
  (10): Sequential(
    (0): Lambda(
    )
    (1): Linear(in_features=2048, out_features=1000)
  )
)

I want to remove the last Linear layer and then add_module but I don’t know how to remove.
Or is it okay to just replace it by,
model[10].add_module('1', torch.nn.Linear(2048,365))?

1 Like
2

Hi

  1. Remove the last layer

removed = list(model.children())[:-1]
model= torch.nn.Sequential(*self.removed)

  1. add the new layer

model = torch.nn.Sequential(model, torch.nn.Linear(2048,365))

12 Likes
3

I have a problem replicating this behaviour in VGG.

It looks to me as ResNet and VGG are constructed differently. I can’t simply: list(model.children())[:-1] because it retrieves an entire nn.Sequential of 6 modules. And I can’t go deeper in the indexing. I retrieve that same Sequential if I try list(model.children())[-1], list(model.children())[:][-1]``` orlist(model.children())[-1]```

The only way I have found possible to do it is:

model = models.vgg16(pretrained=True)
model.name = 'VGG_Baseline'            
        
last = nn.Linear(list(model.children())[-1][-1].in_features, out_size)
block = nn.Sequential(*list(model.children())[-1][:-1], last)
        
model = nn.Sequential(*list(model.children())[:-1], block)

However, this way I can’t use requires_grad_(False) for every layer except this last one.

Any ideas?

2 Likes
4

it doesn’t work. said ‘Sequential’ object has no attribute ‘removed’

6

remove the word “self”

7

Since the source code of this model is:
self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
self.fc = nn.Linear(512 * block.expansion, num_classes)
from pytorch sourcecode in torchvision,
you can simply write:
model = resnext_101_64x4d.resnext_101_64x4d
model.fc = torch.nn.Linear(2048, class_index)

8

Since the source code for VGG is like this:
self.classifier = nn.Sequential(
nn.Linear(512 * 7 * 7, 4096),
nn.ReLU(True),
nn.Dropout(p=dropout),
nn.Linear(4096, 4096),
nn.ReLU(True),
nn.Dropout(p=dropout),
nn.Linear(4096, num_classes),
)
You could simply write:
model.classifier[-1] = nn.Linear(num_of_channels, num_of_classes)