How does .parameter() work?

1

I figured that if I create a list of modules within a model, it seems that when I do model.cuda(), those parameters in the modules in the list are not affected. It seems that .parameter() doesn’t take into account the modules in the list?

1 Like
2

You are correct that it doesn’t. Take a look at nn.ModuleList

3

Hmm, I still don’t quite understand how .parameter() works. What kind of modules does it automatically keep track of?

This sounds like a minor bug that pytorch should consider fixing?

4

.parameter() works like this: it walks all members of the class (anything added to self) and does one of three things with each member:

  1. If the member is a parameter (something registered with register_parameter(...) or of type nn.Parameter), it adds it to the parameters list.
  2. If the member is of type (or is a subclass of) nn.Module, .parameter() is called recursively.
  3. Otherwise, it is ignored

In theory, you could add a 4th option to handle lists, but nn.ModuleList was chosen instead.

5 Likes
5

Ah I see, so if I create a nn.ModuleList insteadl of list, it allows the .parameter() to use option 2 to recursively find the parameters?

Thanks!