Get intermediate states during nn.LSTM forwarding

If I want to look at some intermediate states of LSTM forwarding, like the values of forget gates, input gates etc, are there any simple ways to do it?

(The only solution I can come up with is to rewrite a LSTM)

there isn’t a solution for this. CuDNN does not store intermediate states.