Periodic padding

vision
1

In Pytorch, I cannot find a straightforward possibility to do a convolution (nn.conv2d) with periodic boundary conditions. For example, take the tensor
[[1,2,3],
[4,5,6],
[7,8,9]]
and any 3x3 filter. A convolution with periodic boundary conditions could in principle be done by doing a periodic padding to 5x5
[[9,7,8,9,7],
[3,1,2,3,1],
[6,4,5,6,4],
[9,7,8,9,7],
[3,1,2,3,1]]
and subsequently a convolution with the filter in “padding=0” mode. Unfortunately, I cannot find a padding method that supports this periodic padding.

Is there a simple walkaround?

1 Like
2

You can use nn.ReflectionPad2d for that, or the functional interface F.pad

3

@fmassa, thank you for replying! However, I don’t think that the “ReflectionPad2d” does what I want. The result given by “ReflectionPad2d” for my example is
[[5,4,5,6,5],
[2,1,2,3,2],
[5,4,5,6,5],
[8,7,8,9,8],
[5,4,5,6,5]],
not what I want for the periodic padding.

Periodic padding is a natural choice for problems with periodic boundary conditions in physics.

1 Like
4

I see. If you have a function (say in numpy or scipy) that performs this periodic padding for you, you could somewhat easily write an autograd Function that perform this operation.
That will require writing the backward for this operation, but that can be performed by using index_add function.
Something like (warning: untested)

from autograd import Function
class PeriodicPadding2d(Function):
    @staticmethod
    def forward(ctx, input, pad):
        output = np.periodicpad(input, pad)  # find the function that performs what you want
        ctx.pad = pad
        ctx.size = input.size()
        ctx.numel = input.numel()
        return output

    @once_differentiable
    @staticmethod
    def backward(ctx, grad_output):
        pad = ctx.pad
        idx = grad_output.new(ctx.size)
        torch.arange(0, ctx.numel, out=idx)
        idx = np.periodicpad(idx, pad)  # or whatever is the function
        grad_input = grad_output.new(ctx.numel).zero_()
        grad_input.index_add_(0, idx, grad_output.view(-1))
        return grad_input.view(ctx.size)
5

Could you find it? I need the same function for an assignment.

6

You can use wrap padding in numpy.

import numpy as np
a = np.array([[1,2,3],[4,5,6],[7,8,9]])
b = np.pad(a, (1,1), 'wrap')

b can be used as input for the nn.conv2d

7

I think this will create a bottleneck in GPU based implementations

8

I came across this same problem and it seems that now it is possible to implement periodic boundary conditions using the functional interface F.pad and mode='circular'.

2 Likes