Different masks applied to conv filters at different pixels

I am trying to do convolutions on an image.

However, at different pixels of the image, I want to perform convolution operation at different selected neighbors. So is there a way to use different masks at different pixels?

I know how to apply the same mask for all pixels, which means I can perform convolution operation at same selected neighbors.

One possible solution I have: If there is a way to know where the filter is right now, then I can apply different masks.

Thank you very much!

I think the easiest method would be to apply a manual convolution by unfolding the input and apply the filter to each patch manually.
Here is a small code example.

Thank you very much! But I still don’t quite get it. Let me clarify my question a bit.

Suppose I have a data of size (128,28,28), where 128 is batch size, and 28*28 is size of each sample. When applying convolution, I want use different masked convolution filters at different pixel. Namely, at pixel (1,1), maybe I want to mask out all pixels on its right, but at pixel (2,2), want to mask out all pixels on its left. So do you have any idea here?

Once you’ve unfolded the input and create the patches, you can create the filters in whatever way you have and apply them to the corresponding patch.
In the first part of my code you’ll get all patches, which are stacked in one dimension.
Thereafter you could either loop over the patches and apply the filters or try a batched approach.