Create arbitrary length network with repetitive blocks using nn.ModuleList

1

I want to create a network which has two types of trainable sublayers, with Type1 fixed and shared between Type2.

class Net(nn.Module):      
    def __init__(self):
        super(Net, self).__init__()
        self.Layer1 = nn.Type1
        self.Layer2 = nn.Type2_1
        self.Layer3 = nn.Type1
        self.Layer4 = nn.Type2_2
        .......

How can create this network and define a forward pass for depth K, i.e., the model will have one Type1 layer and K Type2 layers

2

I’m not exactly sure what you are asking for but I’ll try to answer anyway. It is possible to create a nn.ModuleList that contain a list of layers / nn.modules. To do this you could, for example, do something like this

class Net(nn.Module):      
    def __init__(self):
        super(Net, self).__init__()
        self.layers = nn.ModuleList()
        for _ in range(10):
            self.layers.append(nn.Typewhatever)

    def forward(self, x):
        for layer in self.layers:
          # You could do an if isinstance(layer, nn.Type1) maybe to check for types
          x = layer(x)

        return x

Did this help? If you define all the layers that are going to be used in the init you can put them together in the forward pass in any way you’d like, perhaps by checking their types with an if statement. Feel free to further explain your issues and I’ll try to help