It seems that modules provide some functionality through functions of the form module.some_function but also module._some_function (i.e. notice the _underscore_ notation).
For instance module.parameters() vs. module._parameters.
It seems that module.parameters() returns the parameters and module._parameters returns an OrdereredDict() of the actual parameters.
What the difference between the two and their corresponding use cases?
Is that difference clearly documented in the docs? Searching through the Module documentation I couldn’t find anything about ._parameters?
It is a common convention in python to prepend a _foo to a variable name when it is internal and should not be used by the users.
In particular .parameters() is the public API and ._parameters is an internal attribute that you should not use (that is why it is not documented as well).
This makes sense but I’ve seen a lot of code online that uses the non public API and if I’m not mistaken I’ve seen also some pytorch examples uses it as well.
On another note, if you wanted to loop over the params of a module, do some operation and then remove that param, how would you do that with module.parameters?
With module._parameters one could simply use module._parameters.pop(name). Here’s an example of that.
This makes sense but I’ve seen a lot of code online that uses the non public API and if I’m not mistaken I’ve seen also some pytorch examples uses it as well.
Can you link such example please?
On another note, if you wanted to loop over the params of a module, do some operation and then remove that param, how would you do that with module.parameters?
The thing is that the internal API is not recursive (won’t look for parameters in child Modules). So they will give you very different result if you’re not careful!
The right way to do this is to use named_parameter() that will return the name of the attribute as well as the parameter and you can then delattr(mod, name) if you want to remove it.