How to feed data through from an LSTM to a Linear layer

reinforcement-learning
1

I am trying to build a reinforcement-learning system via OpenAI that is going to perform actions such BUY, SELL, HOLD a stock from the market.

From now I am trying to implement an actual NN that will be used by my agent but I have some problems regarding how to shape the date correctly.


class NeuronalNetwork(nn.Module):
    def __init__(self, input_size, stock_env: StockEnv):
        super(NeuronalNetwork, self).__init__()
        self.stock_env = stock_env
        hidden_dim = 256
        input_size = len(self.stock_env.normalized_dataframe.columns)
        self.hidden_size = 128
        self.num_layers = 1
        output_size = self.stock_env.action_space.n
        self.lstm = nn.LSTM(input_size=input_size, hidden_size=self.hidden_size, num_layers=self.num_layers,
                            batch_first=True)
        self.output_layer = nn.Linear(hidden_dim, output_size)
        self.tanh = nn.Tanh()

    def forward(self, x, hidden=None):
        # N x T x D
        # N - the number of windows sizes
        # T - the window size
        # D - the number of indicators and OHLCV in total
        if hidden is None:
            hidden = (
                torch.zeros(self.num_layers, 1, self.hidden_size).to(device),
                torch.zeros(self.num_layers, 1, self.hidden_size).to(device),
            )
        D = len(self.stock_env.normalized_dataframe.columns)
        T = self.stock_env.window_size
        N = 1
        x = x.view(N, T, D).type(torch.FloatTensor).to(device)
        out, (ht, ct) = self.lstm(x, hidden)
        out = self.tanh(out)
        out = self.output_layer(out)
        return out, hidden

I am having a pandas dataframe with my Open, High, Low, Close, Volume price data which is my input_size and my output_size will be 3 (BUY, HOLD, SELL) because I want to predict the next falling action.

I’m not sure what I am doing wrong but I am getting the following error:

mat1 dim 1 must match mat2 dim 0 when I am trying to run self.output_layer(out)

I’m a newbie here, could anyone of you explain please what I am doing wrong?

2

The reason you are getting this error is because the number of input neurons in your linear layer is less than the number of inputs to the layer. You need to change the size to match the output size of your lstm. Can you print the shape of the lstm output doing this

        x = x.view(N, T, D).type(torch.FloatTensor).to(device)
        out, (ht, ct) = self.lstm(x, hidden)
        out = self.tanh(out)
        print(out.shape)
        out = self.output_layer(out)

whatever that outputs you should change the number of input neurons in you linear layer to.