I want to do modify the shape of the convolution layer so that it doesn’t just calculate the up and down direction. It should be something like the deformable convolution but with less flexibility.
For example, I may want to implement a 2D Conv with shape like this:
Do the images show the stride pattern of each conv kernel or are they kernel layouts with valid weights?
In the latter case, you could zero out all invalid weight positions at the initialization as well as their gradients after each backward call.
Yes, they are the kernel layouts.
What do you mean ‘zero out their gradients after each backward call’? Do you mean multiplying a mask to the weights at each iteration?
If you have the mask for the current pattern, you could use it to zero out the initial randomly initialized weight matrix as well as the populated gradients after the backward() call.
Each backward will accumulate the gradients in the param.grad attribute.
Let me know, if you get stuck.
I got it. So this is implemented in the training code in your solution (instead of writen in the model). Thx.
