I learnt of this functionality. For example, we have a VGG16 model:
import torchvision.models as models
model=models.vgg16()
model._modules['classifier'][6] = 1
Sequential(
(0): Linear(in_features=25088, out_features=4096, bias=True)
(1): ReLU(inplace)
(2): Dropout(p=0.5)
(3): Linear(in_features=4096, out_features=4096, bias=True)
(4): ReLU(inplace)
(5): Dropout(p=0.5)
(6): Linear(in_features=4096, out_features=1, bias=True)
)
It turns out you can manipulate the number of input and output features, e.g.
vgg._modules[‘classifier’][6].in_features=5
vgg._modules[‘classifier’][6].out_features=1
(classifier): Sequential(
(0): Linear(in_features=25088, out_features=4096, bias=True)
(1): ReLU(inplace)
(2): Dropout(p=0.5)
(3): Linear(in_features=4096, out_features=4096, bias=True)
(4): ReLU(inplace)
(5): Dropout(p=0.5)
(6): Linear(in_features=5, out_features=1, bias=True)
)
I don’t quite understand how this thing works. If I reduce 1000 neurons to 1, which one is that? And what happens to all the weights? I realize with in_features=5 this wouldn’t work, but with out_features=1 this would. But how?