How to modify the final FC layer based on the torch.model

self.features seems to be empty.
Could you check, that oroginal_model is a valid model?

PS: You can add code with three backticks `. I’ve formatted your code for better readability.

I can do this with ResNet easily but apparently VGG has no fc attribute to call.
If I build:

resnet_baseline = models.resnet50(pretrained=True)
vgg_baseline = models.vgg16(pretrained=True)

I can see the last fc layer of ResNet with resnet_baseline.fc.in_features.
But I just can see the last fc layer with list(vgg_baseline.children())[-1][-1]

I even need the second index because the way the modules are constructed is different from ResNet.

I have attempt to recreate the functionality by:

vgg_baseline.add_module(module=nn.Linear(list(vgg_baseline.children())[-1][-1].in_features, 75), name='fc')

And now I can call vgg_baseline.fc

Any ideas why VGG behaves like that?

Just wanna say more…
suppose use resnet18:

import torchvision.models as models
net = models.resnet18()

as you know the output has 1000 classes:

(fc): Linear(in_features=512, out_features=1000, bias=True)

if you want to change class to 10, someone may do:

net.fc.out_features = 10

I know it’s not working, but if you print the net, it gives you a changed output:

(fc): Linear(in_features=512, out_features=3, bias=True)

that is interesting. However you will get an error when training. The correct approach is:

net.fc = nn.Linear(512, 10)

so, if you want change the input channel from 3 to 1, use:

net.conv1 = nn.Conv2d(1, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)

rather than:

net.conv1.in_channels = 1

That’s it. It bothered me somehow, when I learned PyTorch. Just wanna point out~~~

4 Likes

Hi,
How to set gradients enabled for a particular class in the fc layer?
Ex:
Consider a resnet18 model with 10 classes.
Shape of the fc.weight layer is [10,512]
I want to train the only the weights corresponds to class0.
meaning only one vector of 512 elements.
How to do that?
Thanks

You cannot set the requires_grad attribute on slices of a parameter and would need to zero out the gradients of the frozen part of the parameter.
Alternatively you could also create two parameters (frozen and trainable), concatenate them and use the functional API for the layer operation.
However, the first approach might be a bit simpler.

1 Like

Thank you very much!

First, set the nuew number of output features:

vgg16.classifier[6].out_features = 8

Then, create two new Parameter tensors, one for weights and one for bias:

vgg16.classifier[6].weight = torch.nn.Parameter(torch.randn(7,4096))
vgg16.classifier[6].bias = torch.nn.Parameter(torch.ones(7))

check by running a random image:

im=torch.randn(1,3,224,224)
out=vgg16(im)
>>>tensor([[-19.0846,   2.8862,   4.4280,  -0.4347,   0.8369,  -8.8550,   2.7851]],
       grad_fn=<AddmmBackward>)

1 Like

how do i just simply print out the 4096 from fc3 ?

Assuming the 4096 are used as the number of input features, you could use:

print(model.fc3.in_features)
1 Like

that after training on the new dataset right?

I’m not sure I understand the question.
My code snippet would just print the input features of the fc3 layer. It wouldn’t depend on the training and the new dataset.
Could you explain your current use case a bit and where you’re stuck at the moment?

Would you like to replace this layer with a new one to fit new input shapes?

trying to apply pca on that layer and see what happens.

thanks bro you have solved my problem. op

I seem to be a bit late to the party, but I just wanted to share a supplementary solution for this topic which should work for any VGG:

model = torchvision.models.vgg19_bn(pretrained=True)
n_feats = 1 #   whatever is your number of output features
last_item_index = len(model.classifier)-1
old_fc = model.classifier.__getitem__(last_item_index )
new_fc = nn.Linear(in_features=old_fc.in_features, out_features= n_feats, bias=True)
model.classifier.__setitem__(last_item_index , new_fc)
5 Likes

This worked well for me, thanks!