I have pre-trained a network on a dataset. I wanted to train a linear classifier on a middle layer of it while the weights of the former layers shouldn’t be updated. The new network is going to be trained on another dataset. Can anyone help me with that? I don’t know how to train the classifier in between and how to turn off the gradient update for the first layers.
You can freeze the first few layers. This means you will set the requires_grad parameter to False for those layers. Check this link for more information.
Hope this helps