Automation of Layer Construction

1

I would like to construct a Fully Convolutional Network (FCN) that has 50 (complex) convolution layers. I don’t want to code this layer by layer since each complex convolution layer consists of a normal convolution layer, batch normalization, drop out, max pooling, nonlinearity, and more. So, I like to use a function, let’s call it conv that does this for me; it simply takes kernel_size, and the number of filters, constructs and initializes the layer. Note that, I don’t know what is the shape of the input for each layer. If PyTorch was written in a functional paradigm, I would just feed the output of one layer to next; easy! Though I know how to construct a forward process, I don’t know how to train such net. So, I am facing the following questions:

  1. How would one construct and train a PyTorch net without defining a Net class? And how would one take care of training/testing mode for dropout?

  2. How would one define 50 conv layers semi-automatically while not knowing the shape of the output of each layer?

2

Try taking a look at DenseNet implementations for inspiration (my implementation: https://github.com/kevinzakka/densenet/blob/master/model.py).

Here’s a small snippet that uses a for loop to create multiple user-defined DenseBlocks (which consists in a sequence of BatchNorm - ReLU - Conv). You need to have an equation that calculates the output channels so that you can properly chain the blocks together.

blocks = []
for i in range(num_blocks):
    dblock = DenseBlock(
        num_layers_dense, out_channels, 
        growth_rate, bottleneck, p
    )
    blocks.append(dblock)
    out_channels = dblock.out_channels

self.block = nn.Sequential(*blocks)
1 Like