Freezing CNN Channels

[SudakshK]

Hi,
Can I use the indexing for requires_grad function like layer.weight.data[out_idx, in_idx, :, :].requires_grad = False, to set the channels of my choice to be frozen while training and the weights for these channels must not be updated.

Many Thanks in advanced!

[anantguptadbl (Anant Gupta)]

@SudakshK you can definitely do that

Take a look at this thread for reference

Correct way of freezing layers

Hello, I have a model M and I am cloning it M.clone() Now, I want to freeze certain layers of M.clone(). When I set requires_grad=False, it results in this error: RuntimeError: you can only change requires_grad flags of leaf variables. If you want to use a computed variable in a subgraph that doesn't require differentiation use var_no_grad = var.detach(). How to freeze the layers of M.clone() in that case? I want to ensure that when I backpropagate using the loss computed on a batch using M.…

[SudakshK]

Hi @anantguptadbl thanks for your reply. My problem is little different. I am pruning and rebuilding a model. I want to freeze lets say only one channel of a Conv Layer i.e say the index 0 in the output channels of the Conv layer weights. I am not able to find the solution as the Torch functionality sets the whole tensor gradients to false. One solution I am working on is to freeze the gradients of the tensor to 0 in that particular index. Hence, I don’t want to have whole layer as non-trainable but only a part of that layer.

Thanks in advanced!