How to remove backbone network parametrs from model.parameters?

1

Hi, my model includes a pre-trained network. Now, I am trying to use different learning rate for the backbone network. I can get parameters of the backbone network easily, but how can I get the rest of the parameters. I tried to use remove backbone network parameters from model.parameters but failed. Could you show me how to do that? Thanks.

2

Hi,

One way is to “hide” you backbone from the discovery by putting it in a data structure: self.backbone = [my_backbone]. And since .parameters() does not look into arbitrary data structure it won’t be found.

The other way if you don’t want the weights to be parameters anymore is to make sure the weights are not nn.Parameter() anymore. You can do so by:

weight_tensor = self.weight.detach() # .detach() to extract the Tensor out of the nn.Parameter
del self.weight
self.weight = weight_tensor
3

@albanD Thanks for your answer. Can I create new parameter list via subtracting model.backbone.parameters from model.parameters through a for loop based comparison?

4

They contain the same objects so you can yes: [p for p in model.parameters() if p not in model.backbone.parameters()] or somthing similar.

1 Like