Order of layers in `h` of `LSTM`

1

Hi,

I have a question regarding the hidden state h in LSTM. If I am using a uni-directional LSTM, is h[0] the state for the input layer or for the output layer?

2 Likes
2

In a LSTM, you have 3 inputs: the new signal (“network’s input”), the last hidden state, the last cell state. And you have two outputs: the new cell state (“network’s output”), the new hidden state.

If you write:
new_h, new_c = LSTM(input, [last_h, last_c])
then new_h is the new hidden state, and new_c is the new cell state. The “output layer” is new_c.

I’m not sure if I answer your question.

1 Like
3

I am sorry that I may not clear. My question is as follows.

The first dimension in h of LSTM is num_layers * num_directions. When I use a uni-directional LSTM (num_directions == 1), I want to know if h[0] is the state of the bottom most layer or the top most one.

4

Ah ok, then h[0] would be the state of the first hidden layer (h_0 in the docs). And h(n) the state of the last hidden layer (that you called “output layer” so I did not understand).

1 Like