Same activation at different layers

vision
1

I want to extract the feature maps at different layers of a VGG model.
The VGG model is implemented as a nested sequence of blocks. Each elementary block is a sequence of layers. For example:

block3 = Sequence(
                  Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)),
                 ReLU(inplace)
                 Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                 ReLU(inplace)
                 Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
                 ReLU(inplace)

To extract the activations in block3 for instance, i used the approach detailed here with get_activation(name) function.

for name, module in model._modules.items():
    if name == 'encoder':
        for nname, submodule in module._modules.items():
            if nname == 'block3':
                for layer_name, layer in submodule._modules.items():
                    print(nname, layer)
                    model.encoder.block3.register_forward_hook(get_activation(layer_name))

output = model(x_input)
print(activation.keys())   # dict_keys(['1', '3', '4', '0', '2', '5'])

So far, so good. I can visualize the activations. However, they look exactly the same: when I compute the difference between the output activation of block3.0 and of block3.4, I get 0:

block3_0 = activation['0'].numpy()[0]
block3_4 = activation[4'].numpy()[0]
print(np.sum(block3_0 - block3_4))  # -> 0

Did I setup the forward hook properly so it “attached” to different layers?
Thank you.

2

Could you print layer_name inside the loop and check, if the same name is passed to get_activation, thus overwriting the old activations?

3

Thanks for the reply.
Here is the result of print(layer_name, layer) inside the last loop:

0 Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
1 ReLU(inplace)
2 Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
3 ReLU(inplace)
4 Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
5 ReLU(inplace)

The 5 layers are listed (also shown in the list of keys of activations).

4

You are hooking the output of the whole block3 for each activation, I think this is what you want to do:

for layer_name, layer in module._modules.items():
     print(layer_name, layer)
     model.encoder.block3[int(layer_name)].register_forward_hook(get_activation(layer_name))
1 Like
5

Good catch! That should be the reason for the same activations.

1 Like
6

@spanev , I replaced the nested loop by the loop that you provided. It prints only 3 layers of block3:

0 Conv2d(256, 128, kernel_size=(2, 2), stride=(1, 1))
1 ReLU(inplace)
2 Conv2d(128, 21, kernel_size=(1, 1), stride=(1, 1))

But the architecture of the block is:

self.block3 = nn.Sequential(nn.Conv2d(128, 256, kernel_size=3, stride=1, padding=1),
                                    nn.ReLU(inplace=True),
                                    nn.Conv2d(256, 256, kernel_size=3, stride=1, padding=1),
                                    nn.ReLU(inplace=True),
                                    nn.Conv2d(256, 256, kernel_size=3, stride=1, padding=1),
                                    nn.ReLU(inplace=True))

On the good side, if I compare block3_0 and block3_2, the sum of the difference is > 0.

7

Hmm that’s weird, I just tried it with the block you provided and that’s what I get:

>>> x = torch.rand(1, 128, 64, 64)
>>> block3 = nn.Sequential(nn.Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)),
...                  nn.ReLU(),
...                  nn.Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)),
...                  nn.ReLU(),
...                  nn.Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)),
...                  nn.ReLU())
>>> activation = {}
>>> for layer_name, layer in block3._modules.items():
...     print(layer_name, layer)
...     block3[int(layer_name)].register_forward_hook(get_activation(layer_name))
...
0 Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
<torch.utils.hooks.RemovableHandle object at 0x101aa9ba8>
1 ReLU()
<torch.utils.hooks.RemovableHandle object at 0x10a1452e8>
2 Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
<torch.utils.hooks.RemovableHandle object at 0x101aa9ba8>
3 ReLU()
<torch.utils.hooks.RemovableHandle object at 0x10a1452e8>
4 Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
<torch.utils.hooks.RemovableHandle object at 0x101aa9ba8>
5 ReLU()
<torch.utils.hooks.RemovableHandle object at 0x10a1452e8>
>>> block3(x).shape
torch.Size([1, 256, 64, 64])
>>> activation.keys()
dict_keys(['0', '1', '2', '3', '4', '5'])
8

Can you check if this is the code you have now:

for name, module in model._modules.items():
    if name == 'encoder':
        for nname, submodule in module._modules.items():
            if nname == 'block3':
                for layer_name, layer in submodule._modules.items():
                    print(nname, layer) 
                    model.encoder.block3[int(layer_name)].register_forward_hook(get_activation(layer_name))
9

Great! It works now. All layers of block3 are printed out (I had to restart the notebook kernel).
Thanks a lot!

this_block = model.encoder.block3
for layer_name, layer in this_block._modules.items():
    print(layer_name, layer)
    model.encoder.block3[int(layer_name)].register_forward_hook(get_activation(layer_name))
1 Like