How does the Module subclass maintain its `_modules`?

acgtyrant · May 17, 2017, 6:47am

Suppose I have a Modules class as below:

class Bottleneck(nn.Module):
    def __init__(self, nChannels, growthRate):
        super(Bottleneck, self).__init__()
        interChannels = 4 * growthRate
        self.bn1 = nn.BatchNorm2d(nChannels)
        self.conv1 = nn.Conv2d(nChannels, interChannels, 1, bias=False)
        self.bn2 = nn.BatchNorm2d(interChannels)
        self.conv2 = nn.Conv2d(
                interChannels, growthRate, 3, padding=1, bias=False)

    def forward(self, x):
        out = self.conv1(F.relu(self.bn1(x)))
        out = self.conv2(F.relu(self.bn2(out)))
        out = torch.cat((x, out), 1)
        return out

Then in IPython REPL:

In [6]: a = Bottleneck(1, 2)

In [7]: for b in a.modules():
    print(b)
   ...:     
Bottleneck (
  (bn1): BatchNorm2d(1, eps=1e-05, momentum=0.1, affine=True)
  (conv1): Conv2d(1, 8, kernel_size=(1, 1), stride=(1, 1), bias=False)
  (bn2): BatchNorm2d(8, eps=1e-05, momentum=0.1, affine=True)
  (conv2): Conv2d(8, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
BatchNorm2d(1, eps=1e-05, momentum=0.1, affine=True)
Conv2d(1, 8, kernel_size=(1, 1), stride=(1, 1), bias=False)
BatchNorm2d(8, eps=1e-05, momentum=0.1, affine=True)
Conv2d(8, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)

I read the source code of the Module class, I do not find any code show that how the Module subclass add these Modlue member like “BatchNorm2d” and so on to its _modules.

ruotianluo · May 17, 2017, 8:13am

github.com

pytorch/pytorch/blob/master/torch/nn/modules/module.py#L241




    Returns:
        Module: self
    """
    return self._apply(lambda t: t.type(dst_type))


def float(self):
    """Casts all floating point parameters and buffers to float datatype.


    Returns:
        Module: self
    """
    return self._apply(lambda t: t.float() if not type(t) in torch._integer_tensor_classes else t)


def double(self):
    """Casts all floating point parameters and buffers to double datatype.


    Returns:
        Module: self
    """
    return self._apply(lambda t: t.double() if not type(t) in torch._integer_tensor_classes else t)

acgtyrant · March 28, 2018, 2:52am

Because the code changed already, I can say the answer Module use set_attribue to maintain its _modules.