I’m new to Pytorch. Let’s say we have a 5x5 filter for one conv layer, now I would like to have the shuffled column orders to construct more filters for the layer. How could I do that?
Or put it in this way:
original filter: [c1, c2, c3, c4, c5] #ci is ith column of the filter
shuffled filters: [c2, c1, c4, c3, c5], [c5, c2, c3, c1, c4] …
By doing this, with one set of parameters, we could get several convolved outputs, which could fit some special applications.
Many thanks in advance.
I believe something like this will do the trick.
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
def __init__(self, nshuffles, *args, **kwargs):
self.conv2d = nn.Conv2d(*args, **kwargs)
self.kwargs = kwargs
ncols = self.conv2d.weight.size(2)
self.shuffles = [torch.randperm(ncols) for _ in range(nshuffles)]
def forward(self, input):
outputs = [self.conv2d(input)]
for idx in self.shuffles:
w = self.conv2d.weight
w = w[:, :, idx, :]
b = self.conv2d.bias
outputs.append(F.conv2d(input, w, bias=b, **self.kwargs))
outputs = torch.cat(outputs, dim=1)
nshuffles = 3
in_channels = 3
out_channels = 32
kernel_size = 5
shufconv = ShuffledConv2d(nshuffles, 3, 32, kernel_size, padding=2)
x = Variable(torch.FloatTensor(1, 3, 300, 400))
if __name__ == '__main__':
Hi Nicholas, thanks a lot! It is neat and working.