How to feed a 3D tensor into Neural Network Forward() pass

1

Consider I have a input data

X = torch.Tensor(sample_size,dim1,dim2)

For each data point, its a matrix form with dim1xdim2

Now when I feed into neural network with

nn.Linear(input_Dim,hidden_dim)

I have

output = input.matmul(weight.t())
RuntimeError: size mismatch,

This makes sense because forward only does operation of matrix multiply with a vector

How should I modify my neural net work?

1 Like
2

You can add additional dimensions like this:

batch_size = 10
add_features = 5
in_features = 7
out_features = 12

x = torch.randn(batch_size, add_features, in_features)
lin = nn.Linear(in_features, out_features)
output = lin(x)

Is this what you are looking for?

3

Hi; wondering if I can do the same thing for nn.Conv2D and nn.ConvTranspose2d ?

Right now; I did x = torch.randn(batch_size, add_features, channels, in_features, in_features) but it raises error states that RuntimeError: Expected 4-dimensional input for 4-dimensional weight

4

If you would like to apply the same kernels on add_features, you could push the add_features dimension to the batch dim and apply the standard nn.Conv2d.
Here is a small dummy example with the comparison of the manual approach:

batch_size, add_features, c, h, w = 2, 4, 3, 24, 24

x = torch.randn(batch_size, add_features, c, h, w)
x_ = x.view(batch_size*add_features, c, h, w) # push add_features into batch dim

conv = nn.Conv2d(c, 1, 3, 1, 1, bias=False)
output = conv(x_)
output = output.view(batch_size, add_features, 1, h, w)

# manual approach to verify output
outputs = []
for idx in range(add_features):
    x_part = x[:, idx]
    # get kernel
    kernel = conv.weight
    out = F.conv2d(x_part, kernel, padding=1)
    outputs.append(out)

outputs = torch.stack(outputs)
outputs = outputs.permute(1, 0, 2, 3, 4)

print((outputs - output).abs().max())
> tensor(0., grad_fn=<MaxBackward1>)

Let me know, if that would work for you.

1 Like