I am implementing a convolutional layer with customized convolution operator (such as deformable convolution). I find that I have to extend Pytorch by C and now I want to find the source code of convolution. Or is there any other simpler ways to do it?
I have also tried to do so. Wait for better suggestions.
You could try to use
unfold/fold for a custom convolution. Note that it will be most likely slower than other optimized methods.
I would like to implement a custom convolution layer, where in I actually modify each individual part of the input with other external filter which also requires gradient.
I am succesful in implementing that in python, but it is very slow as expected.
Now to speed it up, I would like to implement the convolution in C++ front end. As shown here https://pytorch.org/tutorials/advanced/cpp_extension.html
But want to use the backward pass of the inbuilt convolution. Also the extra parameter which is a part of the layer should get the gradients as well.
Before starting it, I wanted to know whether I need to write my own backward pass, or does the c++ end take care of it.
Or is there any other simpler way, please suggest.
There is a great reference for understanding Different kinds of Convolution Operators: