# Partially dilated convolution

Hi there.
I currently have following setup:
`nn.Conv1d(IN_SIZE, IN_SIZE, WING * 2 + 1, padding=WING, groups=IN_SIZE)`
Now I want to implement something I coined partially dilated convolution, in essence I want a convolutional layer that has a “dense island” in the middle. Something like this, where 0: represents a gap due to dilation and 1: a kernel `101010101111101010101`. Inference is done on the centre element
I was wondering if you could suggest the simplest way to implement this kind of layer?

Hi Mister Owl!

If I understand your use case correctly, the simplest approach would probably
be to instantiate two `Conv1d`s – one `conv_dense` with `dilation = 1`, and a
second `conv_dilated` with, say, `dilation = 2`. Pass the input to your “partially
dilated convolution” layer through both convolutions and then sum the results.

Some of the weights of `conv_dilated` will overlap with the “dense island”
implemented in `conv_dense` so there will be some redundancy between
the two sets of weight parameters. But that should be okay – most neural
networks have some (perhaps less obvious) redundancy and train and
predict just fine.

(You could also store the left dilated wing, the dense island, and the right
dilated wing in three separate trainable parameters, assemble your
“partially dilated” kernel from them, and then call the functional form,
`torch.nn.functional.conv1d()`, with your assembled kernel. This
would avoid the redundancy and might prove to run faster, but would
be a bit of a nuisance to implement.)

Best.

K. Frank

1 Like

Hi K. Frank,
I will probably just go for the first approach, since I’m short one time, but I will give the second one a try as well, simply because I think it would be a good exercise.

Just to make sure, I understand that approach correctly. One way to achieve this would be to apply a mask to the wings gradients, which zeros every 2nd gradient for the wings (similar to here). Essentially doing manual dilation, right ?

Hi Mister Owl!

I wasn’t imagining doing anything with the gradients (like masking them after
they’ve been computed). Maybe you could make such a scheme work, but it
would seem quite roundabout to me.

I was suggesting encoding, say, the “left wing” in a trainable parameter,
`torch.tensor ([a, b, c, d], requires_grad = True)`, and then
“dilating” it into the relevant portion of the kernel you then pass to `conv1d()`,
`torch.tensor ([a, 0, b, 0, c, 0, d, 0, ...])`. Autograd will track
and its `a`, `b`, `c`, `d` elements.