Access to the weight of an intermediate layer in a network based on a specific input

1

Hi how can I access to the weight matrix based on a specific input image?
my model is like follows structure and I want to access to the fourth feature layer based on an specific image X.
Network(
(features): Sequential(
(0): Conv2d(3, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=5, stride=5, padding=0, dilation=1, ceil_mode=False)
(4): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(5): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(6): ReLU(inplace=True)
(7): MaxPool2d(kernel_size=5, stride=5, padding=0, dilation=1, ceil_mode=False)
(8): AvgPool2d(kernel_size=1, stride=1, padding=0)
)
(classifier): Linear(in_features=512, out_features=2, bias=True)
)
Any help would be very appreciated.

2

Can you do something like Network[4].weight as the nn.Sequential module implements the __getitem__ function?

3

Hi @eqy I tried this;

Network.features[4].weight

But how can I pass X to this structure?

4

I’m not understanding the question. Are you trying to collect the intermediate output of this layer? Are you trying to skip all other layers and just run the layer on the input directly?

For the former I would check out a forward hook example e.g., How can l load my best model as a feature extractor/evaluator? - #6 by ptrblck

For the latter, provided that the convolution’s expected input channels matches that of the input, you can create a new convolution layer and clone the weights of the existing layer into it.

5

I want to do something like that Network(X).features[4].weight but it dose not correct in Pytorch.
Passing input X to the network and then get the weight of feature layer 4.

6

In that case please check the link above and torch.nn.modules.module.register_module_forward_hook — PyTorch 1.13 documentation for more information on how to retrieve the intermediate output.

I assume you mean the intermediate output rather than the weight; .weight refers to the parameters of the model which are not affected by X unless a backward pass is performed after. Their values would not change before/during the forward pass with X.

7

Thank you very much:).