2D input get mat1 dim 1 must match mat2 dim 0

reinforcement-learning
1

I have a PPO agent and give it a tensor of size (1, 128, 100) as input (state).
The state of the environment is a numpy array (128, 100) and then convert to tensor.
The neural network:
state_dim = env.observation_space.shape[0] #128
hidden_dim = 64
action_dim = 20 (discrete)

class Actor(nn.Module):
    def __init__(self, state_dim, hidden_dim, action_dim):
        super(Actor, self).__init__()

        self.action_layer = nn.Sequential(
            nn.Linear(state_dim, hidden_dim),
            nn.Relu(),
            nn.Linear(hidden_dim, hidden_dim),
            nn.Relu(),
            nn.Linear(hidden_dim, action_dim),
            nn.Softmax(dim=-1)).float().to(device)

    def forward(self, state):
        action_probs = self.action_layer(state)
        return action_probs

I got this error:

File "main.py", line 167, in <module>                                                                                                                                     PPO.main(args) action_probs = self.actor(state)                                                                                                                                      File "Anaconda\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl  result = self.forward(*input, **kwargs)                                                                                                                               File "PPO.py", line 125, in forward   action_probs = self.action_layer(state)                                                                                                                               File "Anaconda\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl   result = self.forward(*input, **kwargs)                                                                                                                               File "Anaconda\lib\site-packages\torch\nn\modules\container.py", line 119, in forward      input = module(input)                                                                                                                                                 File "Anaconda\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl   result = self.forward(*input, **kwargs)                                                                                                                               File "Anaconda\lib\site-packages\torch\nn\modules\linear.py", line 94, in forward                                                                                      return F.linear(input, self.weight, self.bias)                                                                                                                        File "Anaconda\lib\site-packages\torch\nn\functional.py", line 1753, in linear                                                                                         return torch._C._nn.linear(input, weight, bias)                                                                                                                     RuntimeError: mat1 dim 1 must match mat2 dim 0

Any advice?
Thanks.

2

Hi Aysuda!

It’s not clear to me what you’re passing into your model.

Please check the shape of the state that gets passed into your
instance of Actor (so then into action_layer). state must be
a tensor of shape [nBatch, 128]; it can’t be a 3-d tensor nor a
tensor of shape [128, some_other_dim].

[Edit: To clarify: state could be a 3-d tensor, just not of shape
[1, 128, 100]. Examples of legal 3-d shapes would be
[1, 100, 128] and [100, 17, 128]. The requirement is that
the trailing dimension match the value of in_features = 128
of the Linear.]

Best.

K. Frank

3

Hi and Thanks for your reply
The state is a numpy array (128, 100) ,and the shape of tensor I’m passing to action_layer is torch.Size([1, 128, 100]). So the input must be [1, 128]?
How do I do this? With Flatten I got this: torch.Size([1, 12800]) and same error.
With torch.rand(1, 128, 100)[ : , : , -1] I got this: torch.Size([1, 128]) but first state is a (128, 100) array and I think it’s a lot of data lost. Am I wrong?

4

Hi Aysuda!

What is the meaning of the value 128 and the value 100?

Linear (in_features = 128, out_features = 64) will apply
an affine transformation to a batch of input vectors of length 128 to
produce a batch of output vectors of length 64. Is your dimension
of length 128 the dimension to which you want to apply such a
transformation?

No, your input doesn’t have to have shape [1, 128] (although
this would be a valid shape for your input). But the trailing dimension
of your input does have to be 128 so that it will match the value
of in_features in your Linear.

In the typical use case input will have shape [nBatch, 128], where
nBatch is the number of samples in your batch.

(A Linear, however, can accept multiple “batch” dimensions, so a
tensor of shape [nBatch, nSomethingElse, 128] would be a valid
input for a Linear with in_features = 128.)

Best.

K. Frank