Questions about global average pooling

[Max_Power]

I have some questions regarding the use of the adaptive average pooling instead of a concatenate. The questions comes from two threads on the forum

Global Average Pooling in Pytorch

I am trying to use global average pooling, however I have no idea on how to implement this in pytorch. So global average pooling is described briefly as: It means that if you have a 3D 8,8,128 tensor at the end of your last convolution, in the traditional method, you flatten it into a 1D vector of size 8x8x128. And you then add one or several fully connected layers and then at the end, a softmax layer that reduces the size to 10 classification categories and applies the softmax operator. Th…

Global average pooling misunderstanding

Hello, l h would like to replace my fully connected layer with global average pooling layer. l have 10 classes and my last convolutional layer output a 3D tensor of 16,25,32. Last_conv=tensor.view(16,25,32) # something to do here !! final_layer=self.global_average_pooling(last_conv) # output=self.Softmax(final_layer) My question is how to pass from 3d tensor of 16,25,32 to 10 (number of classes) through global average pooling ? Thank you

Q1: What is the preferred approach to using global average pooling for current sota models, should there be a fully connected layer after it or have it be a fully convolutional network?

Q2: How do I change the output size to be size k? Do I need to have a conv2d layer before it? From the first forum thread it seems like I need have a layer with k out_channels before the

self.conv2d_last = nn.Conv2d( in_channel, out_channel = k, kernel_size=1)

then in the forward pass I have

x = torch.cat() # if i want to concatenate outputs from different conv layers
x = self.con2d_last
x = F.adaptive_avg_pool2d(x, (1, 1))

Will that get me to a vector of size k that I can use as the output (or will I need to flatten it)? Is this right?

[nile649 (Nilesh Pandey)]

I use fully connected layer,

    self.out1 = nn.Sequential(
        nn.AvgPool2d(4) # where 4 is kernel size

    )





    x = self.conv4(x)
    x = self.out1(x)
    x = x.view(-1, 1024*1*1)

[Naman-ntc (Naman Jain)]

Look at Adaptive average pooling.

Credits : https://github.com/pytorch/vision/issues/538

Hth!

[Max_Power]

So for the adaptive pooling with output size (1, 1), the input is batch x channel x H x W and the output is batch x channel x 1 x 1?

[Naman-ntc (Naman Jain)]

Yeah it will.
You can also use convolutions instead of max pooling, you’ll have to manually create Conv2d later and the fully connected layer instead of average pooling. (As they say then entire network is hyperparameter)