Shuffle the parameters of a filter to construct new filters for one layer

1

I’m new to Pytorch. Let’s say we have a 5x5 filter for one conv layer, now I would like to have the shuffled column orders to construct more filters for the layer. How could I do that?

Or put it in this way:

original filter: [c1, c2, c3, c4, c5] #ci is ith column of the filter

shuffled filters: [c2, c1, c4, c3, c5], [c5, c2, c3, c1, c4] …

By doing this, with one set of parameters, we could get several convolved outputs, which could fit some special applications.

Many thanks in advance.

2

I believe something like this will do the trick.

import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable

class ShuffledConv2d(nn.Module):
    def __init__(self, nshuffles, *args, **kwargs):
        super(ShuffledConv2d, self).__init__()

        self.conv2d = nn.Conv2d(*args, **kwargs)
        self.kwargs = kwargs
        ncols = self.conv2d.weight.size(2)
        self.shuffles = [torch.randperm(ncols) for _ in range(nshuffles)]

    def forward(self, input):
        outputs = [self.conv2d(input)]
        for idx in self.shuffles:
            w = self.conv2d.weight
            w = w[:, :, idx, :]
            b = self.conv2d.bias
            outputs.append(F.conv2d(input, w, bias=b, **self.kwargs))
        outputs = torch.cat(outputs, dim=1)
        return outputs

def main():
    nshuffles = 3
    in_channels = 3
    out_channels = 32
    kernel_size = 5
    shufconv = ShuffledConv2d(nshuffles, 3, 32, kernel_size, padding=2)
    x = Variable(torch.FloatTensor(1, 3, 300, 400))
    print(shufconv(x).size())

if __name__ == '__main__':
    main()
1 Like
3

Hi Nicholas, thanks a lot! It is neat and working.