How to stop updating the parameters of a part of a layer in a CNN model (not the parameters of the whole layer)?

[KFrank]

Hi Monkey!

@MonKeyBoy

there are ten parameters(filters) in a CNN layer, how I can do to only update five of them

If your “ten parameters” are separate Tensors (or Tensors wrapped
in Parameters), set the requires_grad property of the five you wish
to keep unchanged to False (and don’t add them to your optimizer).

If your “ten parameters” are all values in the same Tensor, the best
approach, in my mind, is to store the values before the optimizer update
and then, after the update, restore the values you wish to keep unchanged.

This approach is outlined in the following post:

Set requires_grad according to a binary mask

Hi Megh! There is no way to do what you want – with or without using loops. The requires_grad property applies to the entire tensor, not to elements of the tensor individually. If your use case for this is to “freeze” just certain elements of the Conv2d weight, that is, to only update some of the elements of weight when training, the safest way is to store a copy of the elements prior to calling the update, and then restore them after the update. E.g., something like this: with torch.…

Best.

K. Frank