I found model.named_parameters() will lose the keys and params in my model, but model.state_dict() can not, how to fix this?
I want to use this method to group the parameters according to its name.
Thanks.
for name, param in model.named_parameters():
print name
for k, v in model.state_dict().items():
print k
print type(v)
You should make sure that if you store a list of nn.Parameters, it should be in a nn.ParameterList and not a plain python list, same if you store a list of nn.Modules should be in a nn.ModuleList.
Thanks for the quick reply. My completed code looks like:
param_frozen_list = [] # should be changed into torch.nn.ParameterList()
param_active_list = [] # should be changed into torch.nn.ParameterList()
for name, param in model.named_parameters():
if name == 'frozen_condition':
param_frozen_list.append(param)
elif name == 'active_condition':
param_active_list.append(param)
else:
continue
optimizer = torch.optim.SGD([
{'params': param_frozen_list, 'lr': 0.0},
{'params': param_active_list, 'lr': args.learning_rate}],
lr = args.learning_rate,
momentum = args.momentum,
weight_decay = args.weight_decay)
The key problem is that in the loop result of model.named_parameters(), some layers is lost.
I use model.modules() to check out that they exits in the full module sequences. So is there wrong in my code or a pytorch bug?
the named_parameters() method does not look for all objects that are contained in your model, just the nn.Modules and nn.Parameters, so as I stated above, if you store you parameters outsite of these, then they won’t be detected by named_parameters().
Thanks. I understand what you mean, but I create all the layers by using class LayerName(nn.Module) and there is no any other parameters beyond the torch.nn.layers.
I think they are registered properly and can be visible.
I change the method to group the parameters via model.state_dict() like this:
for k, v in model.state_dict().items():
name = k
param = torch.nn.Parameter(v)
Does this way can be seemed as same with named_parameters()?
Hi,
The thing is that if named_parameters does not work properly, then .parameters doesn’t either, and that could be a problem if you use it to create your optimizer. Check the exemple below of what will and what won’t work.
class MyLayer(nn.Module):
def __init__(self, arg):
super(MyLayer, self).__init__()
self.linear = nn.Linear(...) # that will work
self.linears = nn.ModuleList(nn.Linear(...), nn.Linear(...)) # that will work
self.wrong_linears = [nn.Linear(...), nn.Linear(...)] # that will NOT work
self.wrong_linears_2 = {1: nn.Linear(...), 2: nn.Linear(...)} # that will NOT work
# And similarly for nn.Parameter !
# Also anything saved as `torch.Tensor` or `torch.autograd.Variable` will NOT work
def forward(self, args):
# some code
All the layers I used are in this definition. That’s quite wired. Hmm~
I think that conv_seg layer does not have any parameters so its parameters do not show up…
