Retrieving hidden and cell states from LSTM in a Language model

nlp
5

Thank you for clear explanation. It was very helpful! I just want to confirm one point again:

Since I’m using 2 layer LSTM network, I want to extract hidden state of the last hidden layer. Is the following way of indexing correct way to do:

first_hidden_layer_hidden_state = h_n[0] # torch.Size([1, 1500])
second_hidden_layer_hidden_state = h_n[-1] # torch.Size([1, 1500])

If the above is correct, I’m interested in second_hidden_layer_hidden_state.

Also, is it analogous for the cell state as well?

first_hidden_layer_cell_state = c_n[0] # torch.Size([1, 1500])
second_hidden_layer_cell_state = c_n[-1] # torch.Size([1, 1500])

Thank you!