Conv1d implementation using torch.nn.functional.unfold

1

Hi,

I am trying to implement 1d convolution using the unfold and matrix multiplication operations. Convolution is equivalent to the unfold, matrix multiplication, and fold (or view) operations, and a 2d convolutional layer can be implemented equivalently as follows:

inp = torch.randn(1, 3, 10, 12)
w = torch.randn(2, 3, 4, 5)
inp_unf = torch.nn.functional.unfold(inp, (4, 5)).squeeze()
mat_a = inp_unf.T
mat_b = w.view(w.size(0), -1)
mat_b = mat_b.t()
out_unf = torch.matmul(mat_a, mat_b).T
out = out_unf.view(1, 2, 7, 8)
print((torch.nn.functional.conv2d(inp, w) - out).abs().max())

Unfortunately, this does not appear to generalize to 1d convolutional layers:

inp = torch.randn(1, 5, 10)
w = torch.randn(8, 5, 2)
inp_unf = torch.nn.functional.unfold(inp.unsqueeze(0), 2).squeeze()
mat_a = inp_unf.T
mat_b = w.view(w.size(0), -1)
mat_b = mat_b.t()
out_unf = torch.matmul(mat_a, mat_b).T
out = out_unf.view(...) # Reshape out_unf
print((torch.nn.functional.conv1d(inp, w) - out).abs().max())

as mat_a and mat_b cannot be multiped (36x4 and 10x8). Currently torch.nn.functional.unfold only supports 4D tensors.

Any help is greatly appreciated!

2

Any updates? I’m having the same issue here.

3

This is my implementation for Conv1d that supports groups and bias, you need to unsqueeze input to be 4d


groups = 3
batch = 4
out_channels = 6
in_channels = 6
kernel = 5
bias = torch.rand(out_channels)
inp = torch.randn(batch, in_channels, 1, 12)
w = torch.randn(out_channels, in_channels // groups, 1, kernel)
inp_unf = torch.nn.functional.unfold(inp.view(batch * groups, in_channels // groups, 1, 12), (1, kernel))
# inp_unf.size()   # 10 is kernel * in_channels // groups, 8 is conv output width
# torch.Size([12, 10, 8])
out_unf = inp_unf.view(batch, groups, inp_unf.size(1), inp_unf.size(2)).transpose(2, 3).matmul(w.view(1, groups, out_channels // groups, -1).permute(0, 1, 3, 2)).transpose(2,3)
# out_unf.size()
# torch.Size([4, 3, 2, 8])
e_out = torch.nn.functional.conv2d(inp, w, bias=bias, groups=3)
# e_out.size()
# torch.Size([4, 6, 1, 8])
(out_unf.reshape(batch, out_channels, 1, inp_unf.size(2)) + bias.unsqueeze(0).unsqueeze(2).unsqueeze(3) - e_out).abs().max()
# tensor(4.7684e-07)