Hi,
I want to set different weight decay for each parameter.
I think per-parameter options is related.
But, in my understanding, it can be only applied for each layer or for bias and weights.
I want to know how to set different weight decay for each weight in a layer.
For example, top row is 1e-3 and second row is 1e-4, and bottom row is 1e-3 for weight parameters of a 3x3 convolutional filter.
Can I implement such things in PyTorch?