Group convolution output order

Memory Format
1

I am trying to apply a group 1x1 convolution where one (whole) input is convolved with multiple sets of filters (ngroups), that is: conv = nn.Conv2d(ch, ch*ngroups, 1, groups=ch).

What is the correct way to separate the result of this convolution into respective groups?

conv(input).view(bs, ch, ngroups, h, w), or conv(input).view(bs, ngroups, ch, h, w)

1 Like
2

I think it’s view(bs, ch, ngroups, h, w) based on the following experiment:

bs = = h = w = 1
ch = 3
ngroups = 4

>>> input = torch.zeros(bs, 3, h, w)
>>> input[0,1] = 1

>>> conv = torch.nn.Conv2d(ch, ch*ngroups, 1, groups=ch)

>>> conv.bias
Parameter containing:
tensor([ 0.7412,  0.8724, -0.6242, -0.1312, -0.1991, -0.1502,  0.1400,  0.9860,
        -0.7265,  0.2633,  0.3402, -0.7472], requires_grad=True)

>>> conv(input)
tensor([[[[ 0.7412]], #(=bias)
         [[ 0.8724]], #(=bias)
         [[-0.6242]], #(=bias)
         [[-0.1312]], #(=bias)
         [[ 0.7028]], # not bias -> ch1
         [[-0.3131]], # not bias -> ch1
         [[ 0.0071]], # not bias -> ch1
         [[ 0.6137]], # not bias -> ch1
         [[-0.7265]], #(=bias)
         [[ 0.2633]], #(=bias)
         [[ 0.3402]], #(=bias)
         [[-0.7472]]]], grad_fn=<MkldnnConvolutionBackward>)

>>>conv(input).view(bs, ch, ngroups, 1, 1)[0, 1]
Tensor([[[ 0.7028]],

        [[-0.3131]],

        [[ 0.0071]],

        [[ 0.6137]]], grad_fn=<SelectBackward>)
1 Like