Custom convolutions

1

In standard convolutions, a 3x3 kernel can be thought of as having a center pixel and 8 neighboring pixels which are slid across the entire input image.

I’d like to create a convolution where the kernel shape changes as it is slid across the image. For a target “pixel”, I’d like 4 pixels to be convolved with it: itself, and the 3 pixels it corresponds to if the image is rotated 90, 180, and 270 degrees. The kernel weights stay the same as its shape changes and it is slid across the entire image.

For example, when the kernel starts in the top left, the pixels convolved with it would be top right, bottom right and bottom left.

Is this possible in pytorch? Would I have to modify any C++ code?

2

Hmm… I think I know how.
The image can be rotated and concatenated across the channel axis. Then a 1x1 convolution can be used to achieve the same thing.

3

May it be related to deformable convolutions?
https://pytorch.org/vision/main/generated/torchvision.ops.deform_conv2d.html

1 Like