Using an nn.ModuleList will make sure that all parameters are properly transferred to the device, if you are using model.to() or a data parallel approach.
The attribute should still be found using a plain list object, but should yield a device mismatch error.
Are you trying to create these modules during a forward pass and data parallel?
If so, note that these changes would be applied to each model copy on the device, not the main model.
I would assume the list should also be registered as an attribute, but should yield a device mismatch error. I still don’t know, why the attribute cannot be found at all.
Are you registering it after wrapping the model into a data parallel wrapper?
Sorry for misunderstanding your previous advice before. But no, I register the attribute before warpping the model into data parallel. Here’s my whole script
import torch
import torch.nn as nn
class Example(nn.Module):
def set(self):
list = [1,2,3]
self.__setattr__('exp',list)
def set_torch(self):
conv1 = nn.Conv2d(128,256,3)
self.__setattr__('exp1',nn.ModuleList([conv1,conv1,conv1]))
example = Example()
example.set()
example.set_torch()
example.__getattr__('exp')
example.__getattr__('exp1')
and I get
>>example.__getattr__('exp')
Traceback (most recent call last):
File "E:\pycharm\PyCharm Community Edition 2020.1.3\plugins\python-ce\helpers\pydev\_pydevd_bundle\pydevd_exec2.py", line 3, in Exec
exec(exp, global_vars, local_vars)
File "<input>", line 1, in <module>
File "C:\Users\acer\AppData\Roaming\Python\Python36\site-packages\torch\nn\modules\module.py", line 576, in __getattr__
type(self).__name__, name))
AttributeError: 'Example' object has no attribute 'exp'
>>example.__getattr__('exp1')
ModuleList(
(0): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1))
(1): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1))
(2): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1))
)
Thanks for the code. It seems you can directly access the attribute via example.exp, but example.__getattr_ calls into this derived method, which checks for parameters, buffers, and modules.