Can anyone tell me how to do finetuning in pytorch? Suppose, I have loaded the Resnet 18 pretrained model. Now I want to finetune it on my own dataset which contain say 10 classes. How to remove the last output layer and change to as per my requirement?

Thanks for your reply. I have another doubt here. While fine-tuning, should we not use a smaller learning rate for the whole network ( http://cs231n.github.io/transfer-learning/ )? Here the example says us to fix the network and only learn the weights in the newly added layers? Which one in recommend…

In the snippet I’ve sent you only the last layer is optimized. The rest will be frozen (base parameters will receive no gradients - they have requires_grad=False). This is as if you used learning rate of 0 for that part. If you want to use a lower lr for the base see this section of the optim docs.

Thanks :slight_smile: It helped.

optimizer = torch.optim.SGD([ {'params': model.conv1.parameters()}, {'params': model.bn1.parameters()}, {'params': model.relu.parameters()}, {'params': model.maxpool.parameters()}, {'params': model.layer1.parameters()}, {'params…

This should do it: ignored_params = list(map(id, model.fc.parameters())) base_params = filter(lambda p: id(p) not in ignored_params, model.parameters()) optimizer = torch.optim.SGD([ {'params': base_params}, {'params': model.fc.parameters(), 'lr': opt.l…

This may also help to learn how to modify layers without changing other layers’ parameters and construct a new model. model = models.vgg16(pretrained=True) print list(list(model.classifier.children())[1].parameters()) mod = list(model.classifier.children()) mod.pop() mod.append(…

How do I add new layers to existing pretrained models? Here, the last layer by name is replaced with a Linear layer. How to add another layer after that?

class MyModel(nn.Module): def __init__(self, pretrained_model): self.pretrained_model = pretrained_model self.last_layer = ... # create layer def forward(self, x): return self.last_layer(self.pretrained_model(x)) pretrained_model = torchvision.models.resnet18(pretra…

Thank you for the help. Another question, I am not able to see classifier attribute when loading a pretrained ResNet-18. However, I can directly access the children attribute without needing classifier. Is the classifier attribute part of old version of PyTorch or something I am missing?

PyTorch Forums

How to perform finetuning in Pytorch?

avijit_dasgupta (Avijit Dasgupta) February 12, 2017, 3:33pm 9

Great! Thanks