I am a beginner to PyTorch. I am trying to implement Many-to-Many RNN where the entire input sequence is processed first, and after the last step processing, the output is being generated. Something like this:
For efficiency, I am using pack_padded_sequence. At the end of last time-step (of input sequence) process, I unpack using pad_packed_sequence. Each input sequence length is different. I need to pass on intermediate step’s hidden state outcome to the next timestep to generate output sequence.
This is no problem if I have just one rnn layer. However, I am not able to do this if there are multiple RNN layers. I get intermediate hidden state output of last layer of multi layered RNN. The nn.RNN return the hidden state of outermost RNN layer for intermediate steps. Is there a way to get hidden state output of all RNN layers for intermediate steps?