Get only conv layer variable weights?

1

In a model defined in nn.Sequential way, can we retrieve all the weights in only conv layers, and apply custom regularization to them? and how?

2

you can get or set the weights:

def set_weights(m):
    classname = m.__class__.__name__
    if classname.find('Conv2d') != -1:
        print(m.weight.data)
        m.weight.data.normal_(0.0, 0.02)

# then
sequential_model = sequential_model.apply(set_weights)

but what exactly do you mean by ‘custom regularization’ you mean you’d like to apply some constraint to just those weights?

Is that a good idea? Maybe someone else could chime in? it may be worth trying to apply regularization to other non conv layers as well and see if it improves your model.

3

thanks for the reply!

Yea in paper of neural photo editor, they apply orthogonal regularization on only conv layers and reports improvements. So it prompts me to try!

4

interesting. i asked a buddy of mine (with slightly more experience with exotic models) whether there was any downside. there could be unintended effects but in general its worth giving it a try.

does the above code work for you? is that what you are trying to do? curious to know the result.