Why are 1x1 conv used in SE layers instead of linear layers

[J_Johnson]

A Conv2d 1x1 kernel is identical mathematically to a Linear operation. Please see here for a code demonstration:

Help me rewrite this from tensorflow (Dense layers)

Welcome to the forums! There is a lot to unpack here. When you are calling nn.AdaptiveAvgPool2d(1), you are telling PyTorch that whatever size image comes into that, make it 1x1. So the output of that layer would then be (batch_size, channels, 1, 1). Let’s assume you want that. Then you should call a flatten() afterward. _x = self.global_avg_pool(inputs).flatten(1) When you get down to your convolutional layer, you will need to add those two dims back. _x = self.conv(_x.unsqueeze(2).unsq…