LM by biLSTM(confusing output(not state) order)

Let me ask a quick question:
In my understanding, when concatenating biLSTM output, it should be:

* fh: forward hidden state
* bh: backward hidden state
<char level>
"BAER"
[fh_B; bh_REAB], [fh_BA; bh_REA], [fh_BAE; bh_RE], [fh_BAER; bh_R]

but pytorch’s return look like:

[fh_B; bh_R], [fh_BA; bh_RE], [fh_BAE; bh_REA], [fh_BAER; bh_REAB]

Am I misunderstanding BiLSTM?

I had a similar question; maybe that helps.

1 Like