How to sum every k channels for a CNN feature map

vision
1

Hey guys, I want to sum every k consecutive channels of a variable together. Assume that the input before summing is with the shape like NxCxWxH, and the output after the summation should be Nx(C/k)xWxH. I implemented the method using the code below. the function sum_up is called in the network’s forward function, but it seems to be extremely inefficient. Is there any better way to implement this in pytorch?

def sum_up(input, bottle_neck=2):
      output= []
      for ch in xrange(0, input.data.shape[1], bottle_neck):
           output.append(torch.sum(input[:, ch:ch+bottle_neck, :, :], 1))
      return torch.stack(output).permute(1, 0, 2, 3).cuda()
1 Like
2

You can split the channel dimension with a view/reshape and sum

n,c,h,w = input.size()
res = input.reshape(n, c//k, k, h, w).sum(2)

Best regards

Thomas

4 Likes
3

Thanks! This works for me!

4

Btw, if you are trying to do group sum, it will be available in the next release soon.

1 Like
5

Could you also tell some efficient way of adding channels in an interleaved fashion? For example if there are 6 channels [0-5], I wish to add 0 th and 3rd, 1st and 4th, 2nd and 5th?

6

I have not tried it, but I have a vague idea.

for i in range(No_of_channels):
    new_channel = old_feature_map[i]+old_feature_map[i+1]
   # now concatenate the new channels one after another to get a new feature map.
7

I have implemented this but it seems to be inefficient as we are using for loop. I am actually looking for some efficient way to do that. Anyways thanks for your answer.

8

This should work with advanced indexing like this (haven’t tested it yet):

new_tensor = tensor[:, ::2] + tensor[:, 1::2]

Edit: sorry, you would of course have to specify the middle channel like this:

middle_channel = tensor.size(1)//2
new_tensor = tensor[:, :middle_channel] + tensor[:, middle_channel:]

Sorry for the first part, I misunderstood your goal.

9

Thanks @justusschock for the answer.

10

Did you try it? Does it work?

11

Not yet. Will try it out tomorrow.

12

Well, you can extend the trick above, if you have c channels, c divisible by k*l:

n,c,h,w = input.size()
res = input.reshape(n, c//k//l, k, l, l, h, w).sum(2).view(n, c//k, h, w)

This will sum k channels from an interleaving grid of spacing l.

Best regards

Thomas

1 Like
13

Thanks for the answer @tom

14

Did group sum release yet? Thanks

1 Like