Different weight decay for different layes

1

Hi, is it possible to set different values for different layers or disable the weight decay for some specified layers, thanks

3 Likes
2

It is. Check out the per-parameter options section.

4 Likes
3

Thanks for the reply, it works.

4

@apaszke Is it also possible to make this dynamic? In my case some (mostly) CNN weights are imploding out and the bias parameters are exploding.