Extract features before classification

I need to extract the features before the last layer or before classifying it from this model

can I do this?

PATH = 'focal-base-is224-ws7.pth'
model = FocalTransformer(pretrained=True)
model.load_state_dict(torch.load(PATH),strict=False)
model.head = nn.Identity()

It should work. Did you run into any errors afterward?

As @mxahan said, if model.head is your last layer (the one before which you want to get the get the activations), it should work.

An alternative solution is to register a forward hook to your layer of interest without modifying the rest of the network: torch.nn.modules.module.register_module_forward_hook — PyTorch 1.12 documentation.

i edit the post with the architecture of the model. excuse me if I need the features before the classification layer, should I remove the head only and make avgpool layer be the last layer

Yes, either that or register a forward hook in the avgpool layer.

1 Like