How to freeze layer update?

[spanev (Serge Panev)]

Hi @wonchulSon ,

Please find two threads about it here:

How the pytorch freeze network in some layers, only the rest of the training?

Setting .requires_grad = False should work for convolution and FC layers. But how about networks that have instanceNormalization? Is setting .requires_grad = False enough for normalization layers too?

Correct way to freeze layers

I would like to do it the following way - # we want to freeze the fc2 layer this time: only train fc1 and fc3 net.fc2.weight.requires_grad = False net.fc2.bias.requires_grad = False # passing only those parameters that explicitly requires grad optimizer = optim.Adam(filter(lambda p: p.requires_grad, net.parameters()), lr=0.1) # then do the normal execution of loss calculation and backward propagation # unfreezing the fc2 layer for extra tuning if needed net.fc2.weight.requires_grad = True n…

The second one is doing exactly what @DoubtWang is suggesting above.

Let us know if you need more details about it!