How to freeze the vector at the second last layer of shallow model?

4

You can get the forward activations directly in the model’s forward method e.g. via:

def forward(self, x):
    act1 = self.layer1(x)
    act2 = self.layer2(act1)
    ...
    out = self.last_layer(actX)
    return out

and return it directly as an additional output.
Alternatively, you could also use forward hooks to grab intermediate activations from specified layers as described here.

If you are only interested in the penultimate activation, you could also replace the last layer with nn.Identity without changing the forward at all.