Why do we need to register parameters in pytorch when using nn modules?

1

I was trying to make a custom nn module and I was having issues registering variables. I have been kindly pointed to nn.ModuleList() and torch.nn.ParameterList. However, I think I don’t understand in the first place, why do I need to “register” parameters? Whats the point of all this?

2 Likes
2

In pytorch, we have Variables, which are the building block in autograd, and we have an utility class nn.Parameter, which is used to indicate to nn.Module that that specific variable should be present when .parameters() is called.

For example, you could have something like

class Net(nn.Module):
    def forward(self, input):
        self.x = input * 2
        return self.x

and when you call .parameters(), you don’t want self.x to be returned as a parameter. For more discussion on this matter, have a look at https://github.com/pytorch/pytorch/issues/143

2 Likes
3

whats wrong with returning self.x in self.parameters()?

4

His point is that self.x is a normal attribute, but not a parameter in the computation graph.

2 Likes
5

ah I see so there are special functions like nn.Parameters that register variables so that self.parameters() returns the params (I guess its just a way to organize things, but besides that there doesn’t seem to be any additional special feature about using this) and so that any attribute is not registered automatically/by accident.

1 Like
6

btw what I did notice is that registering None does not mean that .parameters() returns None. So I guess there must be something in the internals of pytorch that makes sure None’s are not returned.

1 Like