Multiple convolution filters with different sizes in parallel

1

Hi,

I want to use multiple convolution filters in parallel with initial weights (I want the filter values to be fixed).

Below is an example of the desired code. Filter lengths are different but the output dimension will be the same due to the padding. In this example, I only used three filters but I would like to use more than a hundred filters. In this case,

  1. What is the best practice to initialize multiple (>100) convolution filters?
  2. And how can I concatenate them all?
    (Let’s assume I already have the filter_weights array)
class Model(nn.Module):
    def __init__():
        super(Model, self).__init__()

        self.filter_weights = [torch.tensor([[[0.1, 0.2, 0.3]]]),
                               torch.tensor([[[0.1, 0.2, 0.3, 0.4, 0.5]]]),
                               torch.tensor([[[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7]]])]

    def forward(x):
        out1 = F.conv1d(x, self.filter_weights[0], padding=1)
        out2 = F.conv1d(x, self.filter_weights[1], padding=2)
        out3 = F.conv1d(x, self.filter_weights[2], padding=3)
        out = torch.cat([out1, out2, out3], dim=1)
        return out
2

If you can come up with the padding logic inside a loop (or use some kind of lookup table), this should work:

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()

        self.filter_weights = [torch.tensor([[[0.1, 0.2, 0.3]]]),
                               torch.tensor([[[0.1, 0.2, 0.3, 0.4, 0.5]]]),
                               torch.tensor([[[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7]]])]

    def forward(self, x):
        out = []
        for idx, filt in enumerate(self.filter_weights):
            padding = idx+1 # try to come up with logic here
            out.append(F.conv1d(x, filt, padding=padding))
        out = torch.cat(out, dim=1)
        return out
1 Like
3

Hi, thanks for your answer. What about efficiency? would they be performed in parallel? I was trying to do something similar with cuda.Streams but have not been able to make it work. I would love to hear your thoughts about it. Thanks in advance.

David

4

Kernels will be put into a queue for execution on the GPU.
Depending on the utilization of the GPU, you might try to use streams, but if a single conv operation is already fully utilizing the device, you most likely won’t see much difference.

1 Like