Adapting 2d gaussian conv weights at each pixel

1

hi,
let’s say we have an image (single plane), and a corresponding matrix V. we want to smooth the image using a 2d gaussian kernel at each pixel (i, j). the gaussian kernel is parametrized with the mean that is the pixel value (i, j) of the image and standard deviation that is the corresponding value in the matrix V at the position (i, j).

the image could be single or minibatch.

  1. is there an efficient way to do this?
  2. can we use one single filter that re-parametrize itself dynamically with respect to V?

thanks

2

This might not be the most efficient way, but you could unfold the image to create image patches (same as the im2col operation), create the filters for all patches (since each patch would need a separate filter), and use a matrix multiplication to compute the output. Once this is done you could reshape the result back to the expected 4-dimensional shape.
Note that this approach would use a lot of memory, since you are explicitly creating the patches and filters.