Extracting and using features from a pretrained model

1

I see a related topic regarding my question here, but i could not find my answer there so i ask it here.

lets say im using the pretrained vgg and i want to extract the features from some specific layers.

Here is what i should do:

# Load the Vgg:
vgg16 = models.vgg16(pretrained=True)
# cut the part that i want:
new_base =  (list(vgg16.children())[:-1])[0]
# if i print the new_base, i will have:
# Sequential(
#   (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
#   (1): ReLU(inplace)
#   (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
#   (3): ReLU(inplace)
#   (4): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), dilation=(1, 1), ceil_mode=False)
#   (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
#   ...
#   ...
#   (30): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), dilation=(1, 1), ceil_mode=False)
# )

# Then here is my feature extractor function:

class FeatureExtractor(nn.Module):
    def __init__(self, submodule, extracted_layers):
        self.submodule = submodule

    def forward(self, x):
        outputs = []
        for name, module in self.submodule._modules.items():
            x = module(x)
            if name in self.extracted_layers:
                outputs += [x]
        return outputs + [x]

Lets say i have my input and i called it Input.
ands I am interested in having the computed features from the
(2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
and
(5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
layers.

Can you please tell me what should i do and how should i do that?
When i call FeatureExtractor, should i do like: FeatureExtractor(new_base,???).
I am not sure how should i the ???, also dont know how to send my infput to the forward function.

My assumption is that because i loaded the pretrained network, i dont have train it anymore, because it already has the pretrain weights, and i can just use them, right?

I also so have second question that is related to this,
is there anyway that i can just do: y = new_base[n](x), where n is the number of layer that i am interested it and x is the input. and get the output y?

Thanks

1 Like
2

I don’t think you can use the vgg16 directly. If you want to using feature maps of some layers in the middle, you may have to rewrite the forward function of the vgg class in the torch vision package, and in that forward function you can return anything you want.

3

Thank you for your answer.
But then i wont be able to use the pretrain weights…
Lets see if pytorch people agrees with you.

4

Sure you can do whatever you want with this model! To extract the features from, say (2) layer, use vgg16.features[:3](input). Note that vgg16 has 2 parts features and classifier. You can call them separately and slice them as you wish and use them as operator on any input.

For the above example, vgg16.features[:3] will slice out first 3 layers (0, 1 and 2) from the features part of model and then I operated the sliced sequence on input.

2 Likes
5

You can still using the pretrained weights, here’s some code:

import torch
import torch.utils.model_zoo as model_zoo
from torchvision.models.vgg import VGG, make_layers, cfg, vgg16

class MyVgg(VGG):

    def __init__(self):
        super().__init__(make_layers(cfg['D']))

    def forward(self, x):
        # here, implement the forward function, keep the feature maps you like
        # and return them
        pass

model = MyVgg()
model.load_state_dict(model_zoo.load_url('https://download.pytorch.org/models/vgg16-397923af.pth'), strict=True)
model(torch.rand(1, 3, 224, 224))
3 Likes
6

Alternatively you could use forward_hooks to achieve this behavior.

2 Likes
8

This is even better : )

9

I am not sure how to do that.
I did this and it gave me an error:

Input = torch.rand(1,3,5,5)

vgg16 = models.vgg16(pretrained=True)
new_base =  (list(vgg16.children())[:-1])[0]

new_base[:3](Input)


Input = torch.rand(1,3,5,5)

vgg16 = models.vgg16(pretrained=True)
new_base =  (list(vgg16.children())[:-1])[0]

vgg16.features[:3](Input)

both of the above cases cause an error and not working, can you please clarify how i should do that

10

Can you please explain what is cfg? and how it can be used?
i printed it and it is like this:

{‘A’: [64, ‘M’, 128, ‘M’, 256, 256, ‘M’, 512, 512, ‘M’, 512, 512, ‘M’], ‘B’: [64, 64, ‘M’, 128, 128, ‘M’, 256, 256, ‘M’, 512, 512, ‘M’, 512, 512, ‘M’], ‘D’: [64, 64, ‘M’, 128, 128, ‘M’, 256, 256, 256, ‘M’, 512, 512, 512, ‘M’, 512, 512, 512, ‘M’], ‘E’: [64, 64, ‘M’, 128, 128, ‘M’, 256, 256, 256, 256, ‘M’, 512, 512, 512, 512, ‘M’, 512, 512, 512, 512, ‘M’]}

But i cannot figure it out what it is saying and how i should use it

11

can you please tell me how i should use it?

12

Oh! This minimal code is working for me.

import torch
import torchvision.models.vgg as models

input = torch.rand(1, 3, 5, 5)
vgg16 = models.vgg16(pretrained=True)
output = vgg16.features[:3](input)
print(output)

I’m using PyTorch 0.4.0 to run this.

5 Likes
13

The cfg is for creating different kinds of VGG nets(like vgg16 or vgg19). Feel free to dig into the source code of vgg.py located at

$YOUR_PYTHON3_DIR/lib/python3.6/site-packages/torchvision/models/vgg.py

It is very clear and simple how to use the cfg after you have a look at this file.
Plus, if you only want to extract feature, and not do any training stuff. @KrnTneja’s solution is better.

1 Like
14

An example towards how to use forward_hooks is given in this post

15

Gotcha!
Gonna switch to version 0.4 :confused:

16

If you don’t want to, you can open ipython and experiment a little to find out what works in your version. (Not the best idea but works! :stuck_out_tongue:)