Get hidden layer activity from a stacked RNN?

1

I want to look at the activity of hidden layers from a multilayered RNN. For example, here’s an implementation from the Pytorch github repo: pytorch rnn example

Having trained the model, we can run the function “evaluate” to test the model on the test set. And I can also collect output and hidden and return them.

According to the pytorch documentation here

  • output has size (seq_len, batch, hidden_size * num_directions)
  • hidden has size (num_layers * num_directions, batch, hidden_size)

… whereas I want to get the activity for all units, all of the layers and all test examples. I thought one of the output dimension should be the number of units in the layer…

Is there a simple way of getting hidden unit activities? Thanks!