How to understand `groups` parameter of `Conv2d`

vision
1

Hello, I am trying to understand parameter groups in Conv2d. I run the following code,

import torch
import torch.nn as nn

m = nn.Conv2d(3,12,3,1,1,groups=3)
fake_in = torch.randn(1,3,244,244)
print(f"input: {fake_in.size()}")
out = m(fake_in)
print(m.weight.size())
# torch.Size([12, 1, 3, 3])
print(out.shape)
# torch.Size([1, 12, 244, 244])

The weight shape is torch.Size([12, 1, 3, 3]), does this weight includes 3 groups? and each group will generate ([1,4,244,244]) in this example?

2

Hi Delpha!

Yes, this is correct.

In effect, you have three groups with in_channels = 1 and
out_channels = 4. The weight tensor for the first group is
m.weight[0:4] of shape [4, 1, 3, 3], the weight tensors for the
second and third groups are m.weight[4:8] and m.weight[8:12],
respectively, also of shape [4, 1, 3, 3]. (Pytorch chooses to store
the three per-group weight tensors as a single tensor of shape
[12, 1, 3, 3] rather than as three separate weight tensors of
shape [4, 1, 3, 3].)

The three per-group convolutions produce three results, each of
shape [1, 4, 244, 244], that are then concatenated together along
dim = 1 to produce the actual result of shape [1, 12, 244, 244].

Best.

K. Frank

3

Thanks, now I understand it.

For this example m = nn.Conv2d(3,12,3,1,1,groups=3), is this called depthwise convolution? or the channel_out must be same with the channel_input?