Convolve over image with small additional tensor

1

Hi all,

I have a question regarding a specific convolution operation I want to perform. Suppose I have some image tensor with shape (1, 12, 256, 256) that I want to convolve with kernel with size 5x5. I can simply do this in PyTorch by doing

import torch
img = torch.randn(1, 12, 256, 256)
conv = torch.nn.Conv2d(in_channels=12, out_channels=32, kernel_size=5, padding=2)
out = conv(img)

Now suppose I have an additional image with the same number of channels and an image size equal to the kernel, i.e., (1, 12, 5, 5).
Now I want the convolution operation to be applied to both images. If they were the same size I could concatenate along the channel dimension into a 24 channel tensor, but as they have different spatial dimensions this operation is not possible. Moreover, padding is not possible because I want the small additional image to be centered on the convolution kernel.

Is there any way to “append” this tensor to every convolution operation?

Thanks.

2

This is possible using F.pad and sounds like a valid approach for your use case:

x = torch.randn(1, 1, 5, 5)
x = F.pad(x, (9, 10, 9, 10))
print(x.shape)
# torch.Size([1, 1, 24, 24])