What does hidden_dim in LSTM do? Is it the size of cell state and hidden state or just the size of hidden state?

1

What does hidden_dim in LSTM do? Is it the size of cell state and hidden state or just the size of hidden state?

2

If you have a look at the source code of LSTMCell, you can see how cell and hidden states are computed in the lines 41 and 42.

cy = (forgetgate * cx) + (ingate * cellgate)
hy = outgate * F.tanh(cy)

Since all operations are element-wise, all gates and states have the same dimension which turns out to be hidden_size.

3

Thanks, I was not sure if LSTM have more than one cell state, which is not mentioned in LSTM documents.