RuntimeError: mat1 and mat2 shapes cannot be multiplied (951x10 and 951x4)

1

Hi I got a error on linear0 function while matrix multiple. Here is the code…

lin = nn.Linear(len(output_from_third), 4)
linear_in = torch.tensor(output_from_third)
print("Linear_in size : {}, Linear_in dim : {}".format(linear_in.size(), linear_in.dim()))

print(lin(linear_in))

and this is output.

Linear_in size : torch.Size([951, 10]), Linear_in dim : 2
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/var/folders/_8/q_cwh5hn0s1dxsrzq2d040p80000gn/T/ipykernel_44889/2866187022.py in <module>
      3 print("Linear_in size : {}, Linear_in dim : {}".format(linear_in.size(), linear_in.dim()))
      4 
----> 5 print(lin(linear_in))

~/miniforge3/lib/python3.9/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

~/miniforge3/lib/python3.9/site-packages/torch/nn/modules/linear.py in forward(self, input)
    101 
    102     def forward(self, input: Tensor) -> Tensor:
--> 103         return F.linear(input, self.weight, self.bias)
    104 
    105     def extra_repr(self) -> str:

~/miniforge3/lib/python3.9/site-packages/torch/nn/functional.py in linear(input, weight, bias)
   1846     if has_torch_function_variadic(input, weight, bias):
   1847         return handle_torch_function(linear, (input, weight, bias), input, weight, bias=bias)
-> 1848     return torch._C._nn.linear(input, weight, bias)
   1849 
   1850 

RuntimeError: mat1 and mat2 shapes cannot be multiplied (951x10 and 951x4)

How do I changed tensor sizes? plz give me some solutions thanks :slight_smile :smile:

2

You need to transpose linear_in in order to be compatible with your nn.Linear layer.

# Example

linear_in = torch.rand(951, 10)
lin = torch.nn.Linear(951, 4)

# This causes an error
lin(linear_in)


# This should work fine
lin(linear_in.T)

As a rule of thumb, the last dimension of your input must match the first dimension of your linear layer.

So if you did something like this, it should also work because the last dim of linear_in is the same as the first of lin.

linear_in = torch.rand(10, 20, 30, 951)
lin = torch.nn.Linear(951, 4)

lin(linear_in)

Hope this helps :smile:

1 Like
3

When your two matrices A1 (m1, n1) and A2 (m2, n2) need to be multiplied, you need to ensure that n1 ==m2!

4

damn, thank you for advise. It’s works. Btw what meaning of .T?

5

.T transposes dimensions 0 and 1 for a 2d tensor.

For a higher dimension tensor you should use torch.transpose(x, dim0, dim1) to specify which dimensions you want to swap. (only changing two dims at the time)

Or if you want to change the order of even more dimensions at the same time, you can use .permute.

Hooe this helps :smile:

1 Like