How can l load my best model as a feature extractor/evaluator?

65

I think it depends on your use case and probably also coding style.
E.g. if I would be working on a new model architecture, where now different features will be returned, I would override the forward. This would make sure I can initialize the model using its new definition without any manipulation on the model itself.
On the other hand, if I just want to check some intermediates e.g. for debugging, I would use hooks as I can directly add them to the model without any changes to it.
Also, I believe that hooks are not scriptable right now.

Your use case might be of course different.

1 Like
66

Thank you for the prompt response!

67

@ptrblck
I wonder about my case. How could I print the output from Unet_Netzero.pretrained.layer1.3.0.act1

Unet_Netzero(
  (quant): QuantStub()
  (pretrained): Module(
    (layer1): Sequential(
      (0): Conv2d(3, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
      (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU(inplace=True)
      (3): Sequential(
        (0): DepthwiseSeparableConv(
          (conv_dw): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32, bias=False)
          (bn1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (act1): ReLU()
          (hardtanh1): Hardtanh(min_val=0, max_val=6, inplace=True)
          (se): Identity()
          (conv_pw): Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn2): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (act2): ReLU()
          (hardtanh2): Hardtanh(min_val=0, max_val=6, inplace=True)
          (skip_add): FloatFunctional(
            (activation_post_process): Identity()
          )
        )
      )

Thanks.

68

Forward hooks would work as given in this thread in e.g. this post.

69

@ptrblck Thanks for the prompt reply.
I still not sure I understand that correctly.
For my case, the DepthwiseSeparableConv is defined in another nn.moduel and didn’t show in the forward() definition of the main network (i.e., Unet_Netzero)

If I want to access pretrained.layer1.3.0.bn1, is the following correct?
model.pretrained.layer1.3.0.bn1.register_forward_hook(get_activation(‘bn1’))

Thanks

70

I’m not sure I understand this question completely. If the module isn’t used in the forward method, there won’t be any forward activations and the hook won’t capture anything.

Yes, looks correct.

71

Hi ptrblck, can we extract any intermediate convolutional layer for feature maps visualisation?

72

Yes, you can register a forward hook on any internal conv layer to get the activation.

73

I see that you have printed the values of print(activation['fc2']). I can read that F.relu() is applied afterwards.
How to get the values of F.relu(self.fc2)?

74

If you want to use forward hooks as well, you could replace the functional F.relu with the nn.ReLU module and register the hook to it.
If not, you can store the activation output of F.relu directly in e.g. a dict inside the forward method.

1 Like
75

Do you have an example of “store the activation output of F.relu directly in e.g. a dict inside the forward method.” I guess you have already explained it somewhere?

76

This should work:

act = {}

class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(1, 1)
        self.fc2 = nn.Linear(1, 1)
        
    def forward(self, x):
        x = F.relu(self.fc1(x))
        act['fc1.relu'] = x.clone()
        x = self.fc2(x)
        return x
    
model = MyModel()
x = torch.randn(1, 1)

print(act)
# {}

out = model(x)
print(act)
{'fc1.relu': tensor([[0.3142]], grad_fn=<CloneBackward0>)}
1 Like
77

Instead of getting the output of an intermediate layer, is there a way to get the input of an intermediate layer? For example if my forward function looks like this, where fc1 and fc2 are just the linear layers:

def forward(self,x,x1):
        x = F.relu(self.fc1(x))
        x_cat = torch.cat((x,x1))
        x = self.fc2(x_cat)
        return x

How can I get x_cat in this case? Thank you!

78

Hi! is there any possibility to do the same with YOLOv7 architecture ? I’d like to get the feature maps of each layer using yolov7 model (below is example of model layers)

utoShape(
  (model): Model(
    (model): Sequential(
      (0): Conv(
        (conv): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True)
        (act): SiLU()
      )

@ptrblck

Thanks!

79

Yes, my previously posted code snippet should work on any model which properly registered submodules.

1 Like
80

what about the loading of the optimizer?
Since there is the new fc layer(because the feature extraction), we can not load the original optimizer, right?
So, what can we do?

81

Yes, this might be the case and you could create an optimizer missing the new fc layer, load its state_dict, and create a separate optimizer for the newly added fc layer.

82

Can we implement this in torchvision resnet50 model without creating a separate class MyModel?

83

Yes, you can use forward hooks in any model using nn.Modules.