How to give input to an intermediate layer

Hi everyone,

Assume that we have a model which takes x as input and outputs y at the end, now I want to train it again but give the input to an intermediate layer rather than at the input of the model. May youp please help me how I need to do it with a simple snippet code.

Thank you in advance,

Hi @Xia_Zhengmi

First of all the intermediate level might not have the same input size as the first level, Iā€™d propose to check the input of your layer_of_interest with pre forward hook that outputs the size of input.

def print_input_size(self, input):
  print('Inside ' + self.__class__.__name__ + 'pre forward')
  print(input.shape)

net.layer_of_interest.register_forward_pre_hook(print_input_size)

Secondly, you can shorten your model using Python slicing and nn.Sequential (this option will work only if your net is already a Sequential, if not find a container in it and slice it)

from = 3 # number of layer_of_interest
new_net = nn.Sequential(*net[from:])

Hope that will help!

Hi @zetyquickly,

Many thanks for your response. Quite helpful and exactly what I need for my project.

1 Like