Convolve over image with small additional tensor

Hi all,

I have a question regarding a specific convolution operation I want to perform. Suppose I have some image tensor with shape (1, 12, 256, 256) that I want to convolve with kernel with size 5x5. I can simply do this in PyTorch by doing

import torch
img = torch.randn(1, 12, 256, 256)
conv = torch.nn.Conv2d(in_channels=12, out_channels=32, kernel_size=5, padding=2)
out = conv(img)

Now suppose I have an additional image with the same number of channels and an image size equal to the kernel, i.e., (1, 12, 5, 5).
Now I want the convolution operation to be applied to both images. If they were the same size I could concatenate along the channel dimension into a 24 channel tensor, but as they have different spatial dimensions this operation is not possible. Moreover, padding is not possible because I want the small additional image to be centered on the convolution kernel.

Is there any way to “append” this tensor to every convolution operation?


This is possible using F.pad and sounds like a valid approach for your use case:

x = torch.randn(1, 1, 5, 5)
x = F.pad(x, (9, 10, 9, 10))
# torch.Size([1, 1, 24, 24])