How to create a directional conv filter

I’d like to create some conv filters with orientation so that they can capture directional informations. The vertical and horizontal filters are easy to implement by changing aspect ratio. But how to create directional filters, such as diagonal?

1 Like

Do you mean, forcing some conv filters to be only diagonal ?

Yes. The alternative solution is rotating feature map manually. I wonder if there are some other simple solutions?

You can create a nn.Parameter w with a size [out_channels, in_channels, kernel_size].
Then torch.diag_embed(w) will have a size [out_channels, in_channels, kernel_size, kernel_size], in which each squared kernel is diagonal.
So you can use nn.functional.conv2d(x, torch.diag_embed(w)).

Thanks for your reply and I have another questions. In the forward, the filter does collect the directional information. But the backward still compute the gradient which change the zeros in original filter. This solution cannot keep the shape of filter?

That is why (in the diagonal case), you can create a parameter w with only the size:
[out_channels, in_channels, kernel_size] (for example : [16, 32, 3]).
This parameter will only contain the useful diagonal information, without the useless zeros. So at the optimization step, only these floats will be updated.

It is just the tensor torch.diag_embed(w) which contains also the zeros. This one has the size:
[out_channels, in_channels, kernel_size, kernel_size] (for example : [16, 32, 3, 3]).

And in the general case, I’m sure you can always find a way to create a parameter with only the useful floats (that need to be updated), and then transform it to the form you need.

Yes thats a general solution! Thanks for your help!~