Why cannot nn.Module use `__setattr__` to set non-pytoch objects?

Recentelly I find nn.Module cannot set some objects as its attributes.for example

class Example(nn.Module):
    def set(self):
        # seq is instance of nn.Sequential
        list = [seq1,seq2,seq3]
        self.__setattr__('exp',list)

when using __getattr__, it will return

AttributeError: ‘Example’ object has no attribute ‘exp’

I figure it out by replacing list with nn.ModuleList but what is the purpose?

Using an nn.ModuleList will make sure that all parameters are properly transferred to the device, if you are using model.to() or a data parallel approach.
The attribute should still be found using a plain list object, but should yield a device mismatch error.
Are you trying to create these modules during a forward pass and data parallel?
If so, note that these changes would be applied to each model copy on the device, not the main model.

1 Like

Thank for your answering and the hint. But what still confuse me is why cannot post a list into model.Any reason for this?

I would assume the list should also be registered as an attribute, but should yield a device mismatch error. I still don’t know, why the attribute cannot be found at all.
Are you registering it after wrapping the model into a data parallel wrapper?

Sorry for misunderstanding your previous advice before. But no, I register the attribute before warpping the model into data parallel. Here’s my whole script

import torch
import torch.nn as nn

class Example(nn.Module):
    def set(self):
        list = [1,2,3]
        self.__setattr__('exp',list)

    def set_torch(self):
        conv1 = nn.Conv2d(128,256,3)
        self.__setattr__('exp1',nn.ModuleList([conv1,conv1,conv1]))

example = Example()
example.set()
example.set_torch()
example.__getattr__('exp')
example.__getattr__('exp1')

and I get

>>example.__getattr__('exp')
Traceback (most recent call last):
  File "E:\pycharm\PyCharm Community Edition 2020.1.3\plugins\python-ce\helpers\pydev\_pydevd_bundle\pydevd_exec2.py", line 3, in Exec
    exec(exp, global_vars, local_vars)
  File "<input>", line 1, in <module>
  File "C:\Users\acer\AppData\Roaming\Python\Python36\site-packages\torch\nn\modules\module.py", line 576, in __getattr__
    type(self).__name__, name))
AttributeError: 'Example' object has no attribute 'exp'

>>example.__getattr__('exp1')
ModuleList(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1))
  (1): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1))
  (2): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1))
)

Thanks for the code. It seems you can directly access the attribute via example.exp, but example.__getattr_ calls into this derived method, which checks for parameters, buffers, and modules.