How to choose adaptive lambda for regularizer?

1

Hai,
Can anyone provide me the way of using adaptive regularizer?
say

reg += torch.sum(torch.mul(w,w)-a)

I need the a to be trainable, how should i do that?
Thanks in advance!

2

If you want a to be a parameter of the model wrap it around nn.Parameter. In your nn.Module add this line: self.a = nn.Parameter(torch.tensor(0.5), requires_grad=True), where 0.5 is the initialization value.