Purpose of SELayer

vision
1

Hi, can anyone tell me, what is the purpose of this Layer. I’m reading mobilenetV3 and GhostNet btw.

class SELayer(nn.Module):
    def __init__(self, channel, reduction=4):
        super(SELayer, self).__init__()
        self.avg_pool = nn.AdaptiveAvgPool2d(1)
        self.fc = nn.Sequential(
                nn.Linear(channel, channel // reduction),
                nn.ReLU(inplace=True),
                nn.Linear(channel // reduction, channel),
            )

    def forward(self, x):
        b, c, _, _ = x.size()
        y = self.avg_pool(x).view(b, c)
        print(y.shape)
        y = self.fc(y).view(b, c, 1, 1)
        print(y.shape)
        y = torch.clamp(y, 0, 1)
        print(y.shape)
        return x * y

Is it some kind of special operation for weighting feature?
Thank you,

2

It looks like the Squeeze-and-Excitation block.

1 Like