How to set a different learning rate for a single layer in a network

9

I am not quite sure what you mean.

As you can see, using the following code (similar to what @ptrblck details in link):

my_list = ['classifier.3.weight', 'classifier.3.bias']
params = list(filter(lambda kv: kv[0] in my_list, vgg16.named_parameters()))
base_params = list(filter(lambda kv: kv[0] not in my_list, vgg16.named_parameters()))

returns two lists - the layer(s) that requires a different learning rate (classifier[3] in this case), and the rest of the network. I am having trouble passing these to the optimizer like so:

optimizer = SGD([{'params': base_params}, {'params': params, 'lr': '1e-4'}], lr=3e-6, momentum=0.9)

I think is incorrect since it gives me errors. Any thoughts @ptrblck, @JuanFMontesinos?

1 Like