Best way to iterate over weights of model

I want to compute L1 regularization. I need to iterate over my network parameters and ignore the bias for computing the L1 loss. Is there a function for checking if a parameter is a weight or bias?

Here is my code. Is this the recommended way? I am afraid that the string comparison in every loop may slow down training.

l1_reg = None
for name, W in net.named_parameters():
    if name[-6:] != 'weight':
        continue
    if l1_reg is None:
        l1_reg = W.norm(1)
    else:
        l1_reg = l1_reg + W.norm(1)

Thanks in advance :D.

I think you almost did it.
A string compare should need almost no computational cost. If you initialize the l1_reg with zero you do not need the second if-closure which also saves a minimal amount of computation time. AFAIK there is no more efficient way to do so.

l1_reg = 0
for name, W in net.named_parameters():
    if 'weight' in name:
        l1_reg = l1_reg + W.norm(1)
3 Likes

the solution you accepted does NOT match the title of the question. Please modify the title or make the body of the question match the title. Having mismatching body and title makes search online harder. Don’t do that.