"if" condition in nn.Sequential

1

I am creating network as below. Is it possible to write “if” condition inside nn.Sequential? I want to make customize if condition is true add nn.LeakyReLU else not.

conv_layers.append(nn.Sequential(nn.Conv2d(3, 5, kernel_size=3, stride=1,padding=1),
nn.LeakyReLU(0.2,True) )

2

Yes, you can use a condition to add a new layer to a list first and then create the nn.Sequential model:

layers = [nn.Conv2d(3, 5, kernel_size=3, stride=1,padding=1)]
cond = True
if cond:
    layers.append(nn.LeakyReLU(0.2,True))
model = nn.Sequential(*layers)
3

It depends on your end goal. If you want a conditional layer in the init function, it will be static during training. (I.e. the condition must be known before training the model and should be a condition that won’t change during training.)

But, if you want the condition to take place during training, you should make the condition during your forward pass. Then it is not possible to place the condition in a nn.Sequential.

I.e.

def forward(cond, x):
    x=self.layer1(x)
    if cond:
        x=self.relu(x)
    ...
    return x