How to manipulate layer parameters by it's names?

I have a complicated CNN model that contains many layers, I want to copy some of the layer parameters from external data, such as a numpy array.

So how can I set one specific layer’s parameters by the layer name, say “conv3_3” ?

In pytorch I get the model parameters via:

params = list(model.parameters())

for p in params:
    print p.size()

But how can I get parameter according to a layer name and then change its values?


What I want to do can be described below:

caffe_params = caffe_model.parameters()

caffe_params['conv3_1'] = np.zeros((64, 128, 3, 3))
5 Likes

you can get the params via: params = model.state_dict() and then they will be a dictionary whose name will be similar to conv3_1

24 Likes

Is it possible obtain objects of type Parameter by name? The use case is to do something like:

optimizer = optim.Adam([param for name, param in model.state_dict().iteritems()
                        if 'foo' in name], lr=args.lr)

but each param here will be a FloatTensor so the optimizer throws a TypeError

2 Likes

Turns out I needed to update my Pytorch version - nn.Module.named_parameters() does the trick

4 Likes

Hi Smth,

Is there some way I can name a layer using a custom name.

Thanks.

Yep. Just self.add_module('your_name', nn.YourModule(params)) in your nn.Module

4 Likes

thank you so much !!!

  • nn.Module.named_parameters() does not contain parameters like batch_norm’s running_mean and running_var
  • nn.Module.parameters() does not contain parameter names
  • model.state_dict() does not contain parameters that we can update requires_grad property

Is there any way that we can list all parameters allowing us to update name, value and requires_grad properties?

26 Likes

The following code is working on PyTorch 1.0-dev with nn.Module.named_parameters(), I think it is a workable work-around for me.

for name, param in model.named_parameters():
    param.requires_grad = False```
1 Like

btw you can also do this (# How to get the module names of nn.Sequential
):

# https://discuss.pytorch.org/t/how-to-get-the-module-names-of-nn-sequential/39682
# looping through modules but get the one with a specific name

import torch
import torch.nn as nn

from collections import OrderedDict

params = OrderedDict([
    ('fc0', nn.Linear(in_features=4,out_features=4)),
    ('ReLU0', nn.ReLU()),
    ('fc1L:final', nn.Linear(in_features=4,out_features=1))
])
mdl = nn.Sequential(params)

# throws error
# mdl['fc0']

for m in mdl.children():
    print(m)

print()

for m in mdl.modules():
    print(m)

print()

for name, m in mdl.named_modules():
    print(name)
    print(m)

print()

for name, m in mdl.named_children():
    print(name)
    print(m)

Just to be sure, this will replace the current module with this new module if the name is the same?

Just wanted to know, have you found any solution to this problem?

For example, to scale a specific layer by a scalar c I did the following:

weight_name = 'fc2.weight'
c = 100
for name, param in model.named_parameters():
    if param.requires_grad:
        if name == weight_name:
            param.data = c*param.data