I am trying to create a Conv2d layer where each batch from the input will be multiply with it’s corresponding kernels. Right now the Conv2d takes as input (N, C_in, W, H) and it has weights of shape (K, K, W, H). I would like to extend the weights to (N, K, K, W, H) so each input will have it’s own set of weights. I tried to manually set the Conv2d.weight.data to the corresponding size but it does not work. Does anyone know how can I achieve this behavior?

Assuming that each input belongs to a different family of data, why don’t you just instantiate several convolutions. What you propose is nothing but stacking convolution outputs, right?

Interesting idea, I did not thought about that. I’m trying to manually set the weights. Isn’t this approach going to be slow? I will need a loop to compute all the results.

Well, for sure the code is very optimized to run as it is. To create such a operation you should rewrite backward function which will be slower.

Anyway, if you want to do that you can just use 3D convolutions with kernel=1 padding= 0 and stride =1 (in the 3rd dimension). That would be exactly what you are asking for.

I am not quite sure I understand your answer. So you would change the input to have the shape (B=1, C_in, Samples. W, H) ? Where Samples is my old batch number and apply 3d conv instead of 2d?

Oh sorry I was wrong, will a kernel = 1 you would be applying the same convolution to each sample and with kernel = B convolutions are not independent. I would go forward stacking 2d convolution outputs