How can i ignore thoes layers?

vision
1

i need to extract features before classifying it … i think i should ignore those layers (avgpool and head)… right? how can i do this

Screenshot from 2021-12-19 15-51-02
Screenshot from 2021-12-19 15-51-021054×551 67.3 KB

2

is there any help please ?

3

You could either use forward hooks to get the desired activation as described here or replace the unwanted layers with nn.Identity modules.

4

thanks a lot for replying but will the results with both methods be the same ? and if I want the features for the images from this model but without a classification layer … should I remove or ignore the last two layers (head and avgpool) … as I tried to ignore them but got the features with large size

5

I’m not sure which both models you mean, so could you explain it a bit more, please?

As previously said, you could replace these layers with nn.Identity or use a forward hook to “skip” them (i.e. get the activation before it’s passed to these two layers). Alternatively, you could of course also write a custom model and remove these layers.

6

thanks . excuse me I tried to extract features from transformer from their GitHub profile by the head. identity() and with using Timm … but I got the different shape for the features from both … using their class I got 768 but using the class where I got 1536 … why this different happen, please ?

7

I don’t know what you are using exactly, but check if some functional API calls are performed in the forward pass and might differ between both approaches.
If that’s not the case, check if you’ve forgotten to remove/replace a pooling layer, which might change the shape etc.