# How to modify the final FC layer based on the torch.model **URL:** \ **Category:** Uncategorized\ **Created:** [February 27, 2017, 12:46pm UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766 "2017-02-27T12:46:27Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![zhongtao93](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/d07c76/32.png) [@zhongtao93](https://discuss.pytorch.org/u/zhongtao93)\ **Post date:** [February 27, 2017, 12:46pm UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/1 "2017-02-27T12:46:28Z")
Hi, everyone. I want to use the VGG19 in my own dataset, which has 8 classes.Ｓo I want to change the output of the last fc layer to 8. So what should I do to change the last fc layer to fit it. Thank you very much! ---
**Author:** ![panovr](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/panovr/32/624_2.png) [@panovr](https://discuss.pytorch.org/u/panovr)\ **Post date:** [February 27, 2017, 12:58pm UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/2 "2017-02-27T12:58:08Z")
You can get the idea from this post [https://discuss.pytorch.org/t/how-to-perform-finetuning-in-pytorch/419](https://discuss.pytorch.org/t/how-to-perform-finetuning-in-pytorch/419) ---
**Author:** ![Ismail\_Elezi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ismail_elezi/32/192_2.png) [@Ismail\_Elezi](https://discuss.pytorch.org/u/Ismail_Elezi)\ **Post date:** [February 27, 2017, 12:59pm UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/3 "2017-02-27T12:59:24Z")
Something like: ``` model = torchvision.models.vgg19(pretrained=True) for param in model.parameters(): param.requires_grad = False # Replace the last fully-connected layer # Parameters of newly constructed modules have requires_grad=True by default model.fc = nn.Linear(512, 8) # assuming that the fc7 layer has 512 neurons, otherwise change it model.cuda() ``` ---
**Author:** ![zhongtao93](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/d07c76/32.png) [@zhongtao93](https://discuss.pytorch.org/u/zhongtao93)\ **Post date:** [February 27, 2017, 1:11pm UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/4 "2017-02-27T13:11:19Z")
Thank you, I’ll try it! ---
**Author:** ![Ismail\_Elezi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ismail_elezi/32/192_2.png) [@Ismail\_Elezi](https://discuss.pytorch.org/u/Ismail_Elezi)\ **Post date:** [February 27, 2017, 1:24pm UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/5 "2017-02-27T13:24:40Z")
Don’t forget to retrain the last layer though. At the moment the weights of it have just random numbers, so you must retrain it. You can do the retraining the same way as normal, the only change is that now the weights of the other layers won’t change because I set the requires\_grad to False. If you have enough data to do a full training, then simply remove the line which does that, but it will surely take much longer to train. ---
**Author:** ![zhongtao93](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/d07c76/32.png) [@zhongtao93](https://discuss.pytorch.org/u/zhongtao93)\ **Post date:** [February 28, 2017, 6:53am UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/6 "2017-02-28T06:53:43Z")
My torch==0.1.9 while torchvision==0.1.7, but the APIs of torchvision.models.vgg19() seems is different from the docs. Here is the [error.It](http://error.It)’s the version problem? I followed the solution from [Pre-trained VGG16](https://discuss.pytorch.org/t/pre-trained-vgg16/664), but it also has problem. ``` --------------------------------------------------------------------------- TypeError Traceback (most recent call last) in () 3 import torch.nn as nn 4 import torch.nn.functional as F ----> 5 model = torchvision.models.vgg19(pretrained=True) 6 for param in model.parameters(): 7 param.requires_grad = False TypeError: vgg19() takes no arguments (1 given) ``` ---
**Author:** ![Crazyai](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/crazyai/32/1355_2.png) [@Crazyai](https://discuss.pytorch.org/u/Crazyai)\ **Post date:** [February 28, 2017, 7:16am UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/7 "2017-02-28T07:16:44Z")
I ran into the same problem. I cloned the latest code repos of `pytorch` and `pytorchvision` and built both manually. Or you can refer to [this issue](https://discuss.pytorch.org/t/pre-trained-vgg16/664) ---
**Author:** ![Ismail\_Elezi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ismail_elezi/32/192_2.png) [@Ismail\_Elezi](https://discuss.pytorch.org/u/Ismail_Elezi)\ **Post date:** [February 28, 2017, 10:10am UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/8 "2017-02-28T10:10:02Z")
Hmm, I didn’t try the code yesterday and just changed the line from a residual network. Now I tried it, and got the same error as you. It seems that you can get just the model, but not the weights of it. Which actually contradicts the documentation when it is written that you can give the pretrained argument. As @Crazyai said, you can refer to that issue and download the model from github. However, bear in mind that they are not planning to release the version with batch normalization. Alternately, use a residual network and finetune it, probably getting even better results than with VGG. ---
**Author:** ![apaszke](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/apaszke/32/21_2.png) [@apaszke](https://discuss.pytorch.org/u/apaszke)\ **Post date:** [February 28, 2017, 12:59pm UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/9 "2017-02-28T12:59:20Z")
Pretrained VGG models are a new thing. Updating **torchvision** to the newest version should fix it. ---
**Author:** ![zhongtao93](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/d07c76/32.png) [@zhongtao93](https://discuss.pytorch.org/u/zhongtao93)\ **Post date:** [March 1, 2017, 1:31am UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/10 "2017-03-01T01:31:29Z")
I uninstalled the current torchvision, and rebulid the newest version, and it worked. Thank you very much! ---
**Author:** ![zhongtao93](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/d07c76/32.png) [@zhongtao93](https://discuss.pytorch.org/u/zhongtao93)\ **Post date:** [March 1, 2017, 2:28am UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/11 "2017-03-01T02:28:05Z")
My network initialized as below: ``` #VGG19 import torchvision import torch.nn as nn import torch.nn.functional as F vgg19 = torchvision.models.vgg19(pretrained=True) for param in vgg19.parameters(): param.requires_grad = False requires_grad=True by default vgg19.fc = nn.Linear(1000, 8) neurons, otherwise change it vgg19.cuda() import torch.optim as optim criterion = nn.CrossEntropyLoss() optimizer = optim.SGD(vgg19.fc.parameters(), lr=0.001, momentum=0.9)#lr 0.001 ``` But when I train with code as below, there is a error in **loss.backward()**: code ``` inputs, labels = Variable(inputs.cuda()), Variable(labels.cuda()) # zero the parameter gradients optimizer.zero_grad() # forward + backward + optimize outputs = vgg19(inputs) #print(type(outputs),type(inputs)) loss = criterion(outputs, labels) #loss = F.nll_loss(outputs, labels) loss.backward() optimizer.step() step += 1 # print statistics running_loss += loss.data[0] ``` error `RuntimeError: there are no graph nodes that require computing gradients` ---
**Author:** ![Cysu](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/cysu/32/113_2.png) [@Cysu](https://discuss.pytorch.org/u/Cysu)\ **Post date:** [March 1, 2017, 3:31am UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/12 "2017-03-01T03:31:27Z")
That’s because vgg19 doesn’t have a `fc` member variable. Instead, it has a ```python (classifier): Sequential ( (0): Dropout (p = 0.5) (1): Linear (25088 -> 4096) (2): ReLU (inplace) (3): Dropout (p = 0.5) (4): Linear (4096 -> 4096) (5): ReLU (inplace) (6): Linear (4096 -> 100) ) ``` To replace the last linear layer, a temporary solution would be ```python vgg19.classifier._modules['6'] = nn.Linear(4096, 8) ``` ---
**Author:** ![zhongtao93](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/d07c76/32.png) [@zhongtao93](https://discuss.pytorch.org/u/zhongtao93)\ **Post date:** [March 1, 2017, 6:38am UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/13 "2017-03-01T06:38:47Z")
Thank you, then how should I change the last layer to **param.requires\_grad = True** ---
**Author:** ![Cysu](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/cysu/32/113_2.png) [@Cysu](https://discuss.pytorch.org/u/Cysu)\ **Post date:** [March 1, 2017, 7:36am UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/14 "2017-03-01T07:36:26Z")
Newly constructed layer has `requires_grad=True` by default. You don’t need to do it manually. ---
**Author:** ![micklexqg](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/micklexqg/32/4748_2.png) [@micklexqg](https://discuss.pytorch.org/u/micklexqg)\ **Post date:** [September 22, 2017, 3:18am UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/15 "2017-09-22T03:18:42Z")
> [@Cysu](#): > > To replace the last linear layer thank you. what should I change if I want to add dropout in ‘fc’? the last classifier layer has been changed in terms of the number of classes and I want to add dropout before it. ---
**Author:** ![chenglu](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/chenglu/32/3901_2.png) [@chenglu](https://discuss.pytorch.org/u/chenglu)\ **Post date:** [March 14, 2018, 1:46pm UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/16 "2018-03-14T13:46:00Z")
There’s already a dropout layer before the final FC layer, the code is ```auto self.classifier = nn.Sequential( nn.Linear(512 * 7 * 7, 4096), nn.ReLU(True), nn.Dropout(), nn.Linear(4096, 4096), nn.ReLU(True), nn.Dropout(), nn.Linear(4096, num_classes), ) ``` you only need to replace the last 4096, num\_classes to your own fc layer. ---
**Author:** ![Praveen\_raja\_sekar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/praveen_raja_sekar/32/13765_2.png) [@Praveen\_raja\_sekar](https://discuss.pytorch.org/u/Praveen_raja_sekar)\ **Post date:** [July 15, 2018, 2:46am UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/17 "2018-07-15T02:46:08Z")
i am trying to convert the classification network to regression netowork by replacing the last layer with number of outputs as one can you please suggest any solution ay help would be greatly appreciated i am new to pytorch ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [July 15, 2018, 8:52am UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/18 "2018-07-15T08:52:19Z")
You can just do exactly this by setting the last layer as: ```python nn.Linear(4096, 1) ``` In your training procedure you probably want to use `nn.MSELoss` as your criterion. ---
**Author:** ![Praveen\_raja\_sekar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/praveen_raja_sekar/32/13765_2.png) [@Praveen\_raja\_sekar](https://discuss.pytorch.org/u/Praveen_raja_sekar)\ **Post date:** [July 15, 2018, 6:21pm UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/19 "2018-07-15T18:21:27Z")
Hi , Thank you for your response. However, i tried that here is my code but the model returns nothing ```python class FineTuneModel(nn.Module): def __init__ (self, original_model, num_classes): super(FineTuneModel, self). __init__ () # Everything except the last linear layer self.features = nn.Sequential(*list(original_model.children())[:-1]) self.classifier = nn.Sequential( nn.Linear(512, 1) ) self.modelName = 'LightCNN-29' # Freeze those weights for p in self.features.parameters(): p.requires_grad = False def forward(self, x): f = self.features(x) f = f.view(f.size(0), -1) y = self.classifier(f) return y model = FineTuneModel(original_model, args.num_classes) print(model) ``` Output: ```python FineTuneModel( (features): Sequential() (classifier): Sequential( (0): Linear(in_features=512, out_features=1, bias=True) ) ) ``` original model is pretrained resnet model ---
**Author:** ![Praveen\_raja\_sekar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/praveen_raja_sekar/32/13765_2.png) [@Praveen\_raja\_sekar](https://discuss.pytorch.org/u/Praveen_raja_sekar)\ **Post date:** [July 15, 2018, 6:22pm UTC](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766/20 "2018-07-15T18:22:09Z")
```python class FineTuneModel(nn.Module): def __init__ (self, original_model, num_classes): super(FineTuneModel, self). __init__ () # Everything except the last linear layer self.features = nn.Sequential(*list(original_model.children())[:-1]) self.classifier = nn.Sequential( nn.Linear(512, 1) ) self.modelName = 'LightCNN-29' # Freeze those weights for p in self.features.parameters(): p.requires_grad = False def forward(self, x): f = self.features(x) f = f.view(f.size(0), -1) y = self.classifier(f) return y ``` [Next page](https://discuss.pytorch.org/t/how-to-modify-the-final-fc-layer-based-on-the-torch-model/766.md?page=2)