How to obtain the tensor from the middle part of a CNN model (EfficientNet_b0), instead of the last layer?

vision
1

How to obtain the tensor from the middle part of a CNN model (EfficientNet_b0), instead of the last layer?

I saw code from some researchers that obtain the tensor from the middle part of ResNet50 in the following method:
To draw ResNet50 in a simple way (there are other ways of drawing it) based on the PyTorch model:
(first 4 structures)
(layer1) (Bottleneck 012)
(layer2) (Bottleneck 0123)
(layer3) (Bottleneck 012345)
(layer4) (Bottleneck 012)
(last 2 structures)
We can obtain the tensor right after layer1,2,3,4 by modifying the lines below and writing our own resnet.py and then load the ResNet50’s pretrained weights:
def forward(self, x): …
x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.layer4(x) …
(from vision/resnet.py at main · pytorch/vision · GitHub)

If I load EfficientNet_b0 (shown in the below EfficientNet_b0 graph) by: efficientnet_b0 = models.efficientnet_b0(pretrained=True) (which means the EfficientNet_b0’s model is already fixed):
(0) (first 3)
(1) (MBConv 0)
(2) (MBConv 01)
(3) (MBConv 01)
(4) (MBConv 012)
(5) (MBConv 012)
(6) (MBConv 0123)
(7) (MBConv 0)
(8)
(last 2)
Is there a simple way of obtaining the tensor right after (5)(6)(7)(8)?

2

You could use forward hooks as described e.g. here or manipulate the forward method directly by creating a new module deriving from the EfficientNet_b0 implementation.

1 Like
3

Thanks. Sorry that I forgot to confirm the solution. Forward Hook is the correct answer to my problem. However, I studied the implementation of Forward Hook from here: Intermediate Activations — the forward hook | Nandita Bhaskhar ,
just in case someone wants to know how to use it, because there’s no much documentation about Forward Hook.