You could use unfold as descibed here to create the patches, which would be used in the convolution.
Instead of a multiplication and summation you could apply your custom operation on each patch and reshape the output to the desired shape.
2 Likes