# Is it mandatory to add modules to ModuleList to access its parameters

**URL:** <https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622>\
**Category:** autograd\
**Created:** [May 17, 2020, 9:44am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622 "2020-05-17T09:44:07Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Abhishek\_Kishore](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/abhishek_kishore/32/12027_2.png) [@Abhishek\_Kishore](https://discuss.pytorch.org/u/Abhishek_Kishore)\
**Post date:** [May 17, 2020, 9:44am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622/1 "2020-05-17T09:44:07Z")

</div>

I read some posts about ModuleList and all of them said that adding modules to ModuleList gives access to parameters of the Neural Network but in “Training a classifier” example of 60 mins pytorch tutorial the modules are not added to any ModuleList and still the parameters could be accessed using

optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)

This is confusing. Please clarify how the parameters are accessible even though the modules have not been added to any ModuleList

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [May 18, 2020, 6:56am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622/2 "2020-05-18T06:56:34Z")

</div>

You don’t need an `nn.ModuleList` to register the parameters properly.  
Could you link the topic, where this is stated, please?

The parameters are registered with the assignment in your model:

```python
class MyModel(nn.Module):
    def __init__ (self):
        super(MyModel, self). __init__ ()
        self.fc = nn.Linear(1, 1) # here all parameters of the linear layers are registered
        self.register_buffer('my_buffer', torch.tensor(1)) # here the buffer is registered

```

The [` __setattr__ ` method](https://github.com/pytorch/pytorch/blob/176174a68ba2d36b9a5aaef0943421682ecc66d4/torch/nn/modules/module.py#L623-L668) in `nn.Module` will take care of it.

Note that plain tensors (not `nn.Parameters`, `nn.Modules` or buffers) will not be registered and will thus not be returned in `model.parameters()` or `model.buffers()`.

---

<div class="post-metadata">

**Author:** ![Abhishek\_Kishore](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/abhishek_kishore/32/12027_2.png) [@Abhishek\_Kishore](https://discuss.pytorch.org/u/Abhishek_Kishore)\
**Post date:** [May 18, 2020, 8:33am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622/3 "2020-05-18T08:33:33Z")

</div>

I am referring to this thread of discusion - [The difference in usage between nn.ModuleList and python list](https://discuss.pytorch.org/t/the-difference-in-usage-between-nn-modulelist-and-python-list/7744)  
In this thread **smth** said

> When you ask for `model.parameters()` , the parameters of layers inside `nn.ModuleList` will be returned. But if it is regular list, we dont look inside the list for `nn.Module` layers.

---

<div class="post-metadata">

**Author:** ![Abhishek\_Kishore](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/abhishek_kishore/32/12027_2.png) [@Abhishek\_Kishore](https://discuss.pytorch.org/u/Abhishek_Kishore)\
**Post date:** [May 18, 2020, 8:37am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622/4 "2020-05-18T08:37:45Z")

</div>

I found one more thread - [When should I use nn.ModuleList and when should I use nn.Sequential? - #15 by Tian\_Jin1](https://discuss.pytorch.org/t/when-should-i-use-nn-modulelist-and-when-should-i-use-nn-sequential/5463/15)

Here you said

> Exactly! If you use a plain python list, the parameters won’t be registered properly and you can’t pass them to your optimizer using `model.parameters()` .

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [May 18, 2020, 8:42am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622/5 "2020-05-18T08:42:24Z")

</div>

Yes, that’s the difference between a Python `list` and an `nn.ModuleList`.  
As explained in the linked topics, the parameters wrapped in a plain `list` won’t be registered, while the parameters from all modules inside an `nn.ModuleList` will be registered.

So if you want to use a list-like container, then the answer to the initial question is: yes, it’s mandatory to use `nn.ModuleList` instead of `list` to register all parameters.

If you don’t need a list-like container and just want to register parameters or layers inside an `nn.Module` class, then you can just use the assignment.

---

<div class="post-metadata">

**Author:** ![Abhishek\_Kishore](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/abhishek_kishore/32/12027_2.png) [@Abhishek\_Kishore](https://discuss.pytorch.org/u/Abhishek_Kishore)\
**Post date:** [May 18, 2020, 9:13am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622/6 "2020-05-18T09:13:43Z")

</div>

Sorry, I am unable to understand the list-like container scenario. Even if a python list contains let’s say 2 linear layers. We would be iterating the list and using those Linear Layers in the forward method of a user defined class(sub classed from nn.Module). Wouldn’t that work?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [May 18, 2020, 9:17am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622/7 "2020-05-18T09:17:59Z")

</div>

It would work to use these layers in the forward pass, but `model.parametes()` will not return the parameters of these layers (and thus `optim.SGD(model.parameters(), lr=1.)` will not see these parameters either).  
Also transferring the parameters to a device via `model.to('cuda')` won’t transfer these parameters to the desired device.

---

<div class="post-metadata">

**Author:** ![Abhishek\_Kishore](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/abhishek_kishore/32/12027_2.png) [@Abhishek\_Kishore](https://discuss.pytorch.org/u/Abhishek_Kishore)\
**Post date:** [May 18, 2020, 9:24am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622/8 "2020-05-18T09:24:08Z")

</div>

So, if a module is added to a python list its parameters get invisible. Thanks for clearly explaining me this concept. I suggest this should be clearly documented with example in pytorch’s documentation of nn.ModuleList.

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [May 18, 2020, 9:27am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622/9 "2020-05-18T09:27:53Z")

</div>

`nn.ModuleList` mentions the advantage here:

> [`ModuleList`](https://pytorch.org/docs/master/generated/torch.nn.ModuleList.html#torch.nn.ModuleList) can be indexed like a regular Python list, but modules it contains are properly registered, and will be visible by all [`Module`](https://pytorch.org/docs/master/generated/torch.nn.Module.html#torch.nn.Module) methods.

If you think this description is still confusing, would you be interested in creating a PR with an improvement?

---

<div class="post-metadata">

**Author:** ![Abhishek\_Kishore](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/abhishek_kishore/32/12027_2.png) [@Abhishek\_Kishore](https://discuss.pytorch.org/u/Abhishek_Kishore)\
**Post date:** [May 18, 2020, 9:31am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622/10 "2020-05-18T09:31:27Z")

</div>

Yes sure. I have never contributed to any open source yet even if it were some documentation.

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [May 18, 2020, 9:33am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622/11 "2020-05-18T09:33:21Z")

</div>

Then this is the best time to start doing it! 🙂  
You could create a feature request [here](https://github.com/pytorch/pytorch/issues), explain your confusion and your suggestion, and wait until the module owners start the discussion.

---

<div class="post-metadata">

**Author:** ![Abhishek\_Kishore](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/abhishek_kishore/32/12027_2.png) [@Abhishek\_Kishore](https://discuss.pytorch.org/u/Abhishek_Kishore)\
**Post date:** [May 18, 2020, 9:44am UTC](https://discuss.pytorch.org/t/is-it-mandatory-to-add-modules-to-modulelist-to-access-its-parameters/81622/12 "2020-05-18T09:44:55Z")

</div>

I created an issue #38639