How to implement a neural network with 2-d tensor as input and output?

Hi,

I am trying to train a simple NN which takes 2-d tensor as input data, and outputs a 2-d tensor. Specifically, I would like to have an input and output of shape 16x2.

Below is the model I used.

class MyModel(nn.Module):
    
    def __init__(self): 
        super(MyModel, self).__init__()

        self.fc1 = nn.Linear(16, 256)
        self.fc2 = nn.Linear(256, 512)
        self.fc3 = nn.Linear(512, 256)
        self.fc4 = nn.Linear(256, 16)
        
    def forward(self, x):
        
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = F.relu(self.fc3(x))
        x = F.sigmoid(self.fc4(x))
        
        return x

With a batch size of 1, I get this error message:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (16x2 and 16x256)

And, with a batch size of 128 (my preferred option is this, as I need to train my model), I get this error message:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (2048x2 and 16x256)

Would you please help me how to use a batch of tensors with dimensions 16x2 as input and having a batch of tensors with dimensions 16x2 as output.

Obviously, if batch size is 128, input and output dimensions should be 128x16x2.

Best wishes…

Hi,
You are facing this error because your first linear layer expects an input vector of size 16 while you are using as input a tensor of dimension batch_size * 16 * 2.

Either change the dimensions of your input tensor or have a look at other layers like convolutional layers in case you are working with images.

Hi,

Thanks for your prompt response.

The thing is I would like to have the outputs of all layer’s activations as 2d tensor (ignoring the batch size for now).

For example, when I fed the input (16x2), I want the dimensions of first layer’s output as 256x2, second layer’s output as 512x2, third layer’s output as 256x2 and output layer output as 16x2.

I hope it is clear now.

I guess it is not possile with nn.linear , right ? any other possibility ?

I am not sure what your use case in terms of the type of data and modelling problem is.
But, you could try reshape/view to change your 1 dimensional tensor to 2 dimensional.

Feel free to post further queries.

Hi,

I already use an input as 2d with shape (16x2) when feeding to the model. (ignoring batch size)

when I feed my input to the model above, I got below error.
dimensions of my input are batch_size x (16x2). batch size is 128.

Input In [5], in MyModel.forward(self, x)
     14 def forward(self, x):
     16     print("shape of x initially is ", x.shape) #shape of x initially is  torch.Size([128, 16, 2])
---> 18     x = F.relu(self.fc1(x))
     19     x = F.relu(self.fc2(x))
     20     x = F.relu(self.fc3(x))

File ~\Anaconda3\envs\pytorchenv\lib\site-packages\torch\nn\modules\module.py:1102, in Module._call_impl(self, *input, **kwargs)
   1098 # If we don't have any hooks, we want to skip the rest of the logic in
   1099 # this function, and just call forward.
   1100 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102     return forward_call(*input, **kwargs)
   1103 # Do not call functions when jit is used
   1104 full_backward_hooks, non_full_backward_hooks = [], []

File ~\Anaconda3\envs\pytorchenv\lib\site-packages\torch\nn\modules\linear.py:103, in Linear.forward(self, input)
    102 def forward(self, input: Tensor) -> Tensor:
--> 103     return F.linear(input, self.weight, self.bias)

File ~\Anaconda3\envs\pytorchenv\lib\site-packages\torch\nn\functional.py:1848, in linear(input, weight, bias)
   1846 if has_torch_function_variadic(input, weight, bias):
   1847     return handle_torch_function(linear, (input, weight, bias), input, weight, bias=bias)
-> 1848 return torch._C._nn.linear(input, weight, bias)

RuntimeError: mat1 and mat2 shapes cannot be multiplied (2048x2 and 16x256)

if, for example, my batch size is 128 as in my case, my requirement is below:

input shape: 128 x 16 x 2
1st hidden layer output shape: 128 x 256 x 2
2nd hidden layer output shape: 128 x 512 x 2
3rd hidden layer output shape: 128 x 256 x 2
output layer shape: 128 x 16 x 2

I wonder how to design and implement such a NN.

Best wishes,

Hi,

This is expected as

Please see if this helps:

class Model(nn.Module):
  def __init__(self):
    super(Model, self).__init__()
    self.fc1 = nn.Linear(32, 512)
    self.fc2 = nn.Linear(512, 1024)
    self.fc3 = nn.Linear(1024, 512)
    self.fc4 = nn.Linear(512, 32)

  def forward(self, x):
    x = x.reshape(-1, 32) # reshaping the input tensor to (batch_size * 32)
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = F.relu(self.fc3(x))
    x = F.sigmoid(self.fc4(x))
    x = x.reshape(-1, 16, 2)
    return x

inp = torch.rand(128, 16, 2)     
net = Model()
out = net(inp)
print(out.shape) # torch.Size([128, 16, 2])

You could use similar reshape calls to change the dimensions of hidden layer tensors if you require to use these intermediate outputs in any way.

Best,
S

Thanks for your help.

But the thing 2-d data is my strict requirement. I have to solve this issue without reshaping to 1d tensor first and reshaping to 2d tensor at the end.

Again, if, for example, my batch size is 128 as in my case, my requirement is below:

input shape: 128 x 16 x 2
1st hidden layer output shape: 128 x 256 x 2
2nd hidden layer output shape: 128 x 512 x 2
3rd hidden layer output shape: 128 x 256 x 2
output layer shape: 128 x 16 x 2

I want my NN to handle this 2d data throughout the whole feedforward and backforward passes.

In that case, too, I think you will have to permute your tensor dimensions as for the nn.Linear layer to be able to process multi-dimensional inputs, the last dimension of the input tensor must match the in_features and the output tensor is then the same shape as the input tensor but for the last dimension which is equal to out_features.

So, input 128 x 16 x 2 cannot be mapped to 128 x 256 x 2.
Although mapping 128 x 2 x 16 to 128 x 2 x 256 is certainly possible.

If it satisfies your use case, you could use torch.permute to re-arrange your tensors like so:

import torch
inp = torch.rand(128, 16, 2)
inp = torch.permute(inp, (0, 2, 1))
inp.shape # torch.Size([128, 2, 16])

and further redefine your network. See:

fc1 = torch.nn.Linear(16, 256)
out = fc1(inp)
print(out.shape) # torch.Size([128, 2, 256])
1 Like

Just out of curiosity, why?

Hi @jarvico,

Following on from @srishti-git1110 answer, you can get the desired behaviour by simply transposing and exploit broadcasting within a nn.Linear layer.

import torch
from torch import nn, Tensor


class model(nn.Module):

  def __init__(self):
    super(model ,self).__init__()
    layers = []
    hid_in, hid_out = [16, 256, 512, 256], [256, 512, 256, 16]
    for hin, hout in zip(hid_in, hid_out):
      layers.append(nn.Linear(hin, hout))
    self.layers = nn.ModuleList(layers)

  def forward(self, x: Tensor) -> Tensor:
    for i, layer in enumerate(self.layers):
      x = layer(x.transpose(-2,-1)).transpose(-2,-1) #transpose to exploit broadcasting over 2d-input
      print(f"Layer {i}, size: {x.shape}")
    return x

x=torch.randn(128,16,2)

net = model()
net(x)

This returns

Layer 0, size: torch.Size([128, 256, 2]) #hidden 1
Layer 1, size: torch.Size([128, 512, 2]) #hidden 2
Layer 2, size: torch.Size([128, 256, 2]) #hidden 3
Layer 3, size: torch.Size([128, 16, 2]) #output 
1 Like

Thank you very much, this works!