Adjust optimizer learning rate

vision
1 
def op_copy(optimizer):
    for param_group in optimizer.param_groups:
        param_group['lr0'] = param_group['lr']
    return optimizer

lr = 1e-3
param_group = []
for k, v in net.named_parameters():
    param_group += [{'params': v, 'lr': lr * 0.1}]
for k, v in classifier.named_parameters():
    param_group += [{'params': v, 'lr': lr}]

optimizer = optim.SGD(param_group)
optimizer = op_copy(optimizer)

Hi, all.
I’m new to pytorch.
I have some questions about the above codes.

  1. What does op_copy() do?

  2. If I change
    optimizer = optim.SGD(param_group) to
    optimizer = optim.SGD(param_group, lr = 1e-3)
    then the learning rate for net become 1e-4 and classifier for 1e-3 ?

2

  1. It seems a new key named lr0 is added the all param groups. If I’m not mistaken, this key is not used in PyTorch optimizers so I guess your script might be using it somewhere manually.

  2. Yes, that should be the case.