Hello, I want to implement Conv2d with different kernels for each pixel. Is there any way to make it in pytorch and keep it as efficient as regular convolution?
Yes, you could start by implementing the desired work flow using e.g. nested for loops and once it’s working use unfold to apply a matmul to speed it up, if that fits your use case.
That won’t be straightforward, but you could e.g. write a custom CUDA extension for your convolution to get another speedup.
1 Like