Access the weights before last FC layer of a trained model

So, I have a trained classification model. Which works pretty well, given any unknown image. We can use class activation map to see which features are getting activated to classify a given object.

In case of binary classification, some features play role to decide ‘0’, some play role to decide ‘1’. This features/weights are decided just before the fully connected classification layer, per my understanding.

  1. so In case of densenet169 trained model, How can I get access to those weights before the last FC layer ?
  2. Can you give any idea/code/examples to distinguish the deciding features and plot them which are contributing to make decision.

Thanks in advance.

Hi @Rosa,

Weights/parameters and features/activations are (usually) not the same thing. The features are (usually) the result of previous weights op features where op can be any binary operator: in Python, weights are contained in the operators and features are the results of operators.
In both cases you will be able to access them with hooks.

In the case of a loaded trained densenet169, you should first identify the classifier Linear layer by printing the architecture:

net = models.densenet169()
print(net)

which seems to be

net.classifier
Linear(in_features=1664, out_features=1000, bias=True)

If you wanna look into the input features and/or weights, your hook should look like this:

global classifier_input_features # to be able to access it anywhere
global classifier_weights
def hook_fn(module, input, output):
    global classifier_input
    global classifier_weights

    classifier_input_features = input
    classifier_weights = module.weight.data

     # you can use matplotlib or TensorboardX to visualize them

hook = net.classifier.register_forward_hook(hook_fn)

after that, yo can access classifier_input_features and classifier_weights after the forward pass.

1 Like

Thanks, for clear explanation. I was mixing up those terms. Also thanks for the code snippet. However I was already able to extract both weights and features.

But can you give me a idea how to distinguish the factors from which a classifier taking the decision to tell a given input either o or 1 ?