When should I use nn.ModuleList and when should I use nn.Sequential?

vision
20

The question was when, and the answer may be in cases when you need dynamic module structure and you don’t know in advance how it will look.
The original example provided is fair:

class MyModule(nn.Module):
    def __init__(self):
        super(MyModule, self).__init__()
        self.linears = nn.ModuleList([nn.Linear(10, 10) for i in range(10)])

    def forward(self, x):
        # ModuleList can act as an iterable, or be indexed using ints
        for i, l in enumerate(self.linears):
            x = self.linears[i // 2](x) + l(x)
        return x

This doesn’t have anything with dynamic graph creation, which PyTorch also do.

If I am not wrong, there must be at least one forward method in PyTorch, so module list will be part of class which will evaluate in that class forward.

Using a class derived from nn.Module some call also a functional approach.

nn.ModuleList can be child of nn.Sequentional and in that case inside sequentional we need to have a class that aggregates it.