nn.MaxPool2D layer

torch.package / torch::deploy
1

Since batchnorm layer gathers statistics during the training step and reuse them later during inference, we have to define a new batchnorm layer every time it is used. The question is if this also applies to maxpooling or is it enough to define it once and use multiple times
if we take the follwing script as example is it enough to only use max_pooling1 layer for both usages
thnks

self.conv1 = nn.Conv2d(out_channel_4, out_channel_5, kernel_size=(9, 9), bias=True, stride=1, padding='same')
self.batch_Norm1 = nn.BatchNorm2d(out_channel_3, momentum=momentum)
self.conv2 = nn.Conv2d(out_channel_3, out_channel_4, kernel_size=(9, 9), bias=True, stride=1, padding='same')
self.batch_Norm2 = nn.BatchNorm2d(out_channel_4, momentum=momentum)
self.max_pooling1 = nn.MaxPool2d(2, stride=2)
self.max_pooling2 = nn.MaxPool2d(2, stride=2)

    def forward(self, x1):
        # block 1.1
        x1 = self.conv1(x1)
        x1 = self.batch_Norm1(x1)
        x1 = F.relu(x1)
        x1 = self.max_pooling1(x1)
        x1 = self.conv2(x1)
        x1 = self.batch_Norm2(x1)
        x1 = F.relu(x1)
        x1 = self.max_pooling2(x1)
2

Correct mean and variance of the whole dataset needed to do proper normalization, but batchnorm has access only to batch part of data each step. That’s why batchnorm is saving intermediate values and updates them each batch-step.

Maxpool only returns max value in the pool in input data. Thus, you don’t need to define different maxpooling instances if they have same parameters.

1 Like