How to manipulate layer parameters by it's names?

#1

I have a complicated CNN model that contains many layers, I want to copy some of the layer parameters from external data, such as a numpy array.

So how can I set one specific layer’s parameters by the layer name, say “conv3_3” ?

In pytorch I get the model parameters via:

params = list(model.parameters())

for p in params:
    print p.size()

But how can I get parameter according to a layer name and then change its values?

What I want to do can be described below:

caffe_params = caffe_model.parameters()

caffe_params['conv3_1'] = np.zeros((64, 128, 3, 3))
5 Likes
#2

you can get the params via: params = model.state_dict() and then they will be a dictionary whose name will be similar to conv3_1

23 Likes
#3

Is it possible obtain objects of type Parameter by name? The use case is to do something like:

optimizer = optim.Adam([param for name, param in model.state_dict().iteritems()
                        if 'foo' in name], lr=args.lr)

but each param here will be a FloatTensor so the optimizer throws a TypeError

2 Likes
#4

Turns out I needed to update my Pytorch version - nn.Module.named_parameters() does the trick

4 Likes
#5

Hi Smth,

Is there some way I can name a layer using a custom name.

Thanks.

#6

Yep. Just self.add_module('your_name', nn.YourModule(params)) in your nn.Module

3 Likes
#7

thank you so much !!!

#8
  • nn.Module.named_parameters() does not contain parameters like batch_norm’s running_mean and running_var
  • nn.Module.parameters() does not contain parameter names
  • model.state_dict() does not contain parameters that we can update requires_grad property

Is there any way that we can list all parameters allowing us to update name, value and requires_grad properties?

27 Likes
#9

The following code is working on PyTorch 1.0-dev with nn.Module.named_parameters(), I think it is a workable work-around for me.

for name, param in model.named_parameters():
    param.requires_grad = False```
#10

btw you can also do this (# How to get the module names of nn.Sequential
):

# https://discuss.pytorch.org/t/how-to-get-the-module-names-of-nn-sequential/39682
# looping through modules but get the one with a specific name

import torch
import torch.nn as nn

from collections import OrderedDict

params = OrderedDict([
    ('fc0', nn.Linear(in_features=4,out_features=4)),
    ('ReLU0', nn.ReLU()),
    ('fc1L:final', nn.Linear(in_features=4,out_features=1))
])
mdl = nn.Sequential(params)

# throws error
# mdl['fc0']

for m in mdl.children():
    print(m)

print()

for m in mdl.modules():
    print(m)

print()

for name, m in mdl.named_modules():
    print(name)
    print(m)

print()

for name, m in mdl.named_children():
    print(name)
    print(m)
#11

Just to be sure, this will replace the current module with this new module if the name is the same?

#12

Just wanted to know, have you found any solution to this problem?