Output of Multilayer Bidirectional RNN

#1

Hi all,

the output of RNN is: h_n (num_layers * num_directions, batch, hidden_size) and output (seq_len, batch, hidden_size * num_directions).

If I would like to use the toppest layer of the h_n, i.e., (num_directions, batch, hidden_size), how should I slice the h_n?

It seems that:
v1 = h_n.view(num_layers,num_directions,batch,hidden_size)
and
v2 = h_n.view(num_directions,num_layers,batch,hidden_size).permute(1,0,2,3)

are not same. Which one should I use to slice the toppest layer?

Thanks!

1 Like
#2

If you want to get the top layer, you could return the first output of nn.LSTM:

lstm = nn.LSTM(3, 3, num_layers=2, batch_first=True, bidirectional=True)
x = Variable(torch.rand(1, 5, 3)) # batch=1, seq_len=5, dim=3
h, (h_t, c_t) = lstm(x)
print h.size() # (batch, seq_len, dim) = (1, 5, 6)
h_n = h[:,-1,:] # get the last time step, return (batch, dim) = (1, 6)
2 Likes
#3

Thanks. It solved my question.