Here is the code:
import torch
from torch.nn.utils.rnn import pack_sequence, pad_packed_sequence
rnn = torch.nn.LSTM(2, 3, 1, bidirectional=True)
x1= torch.randn(1,2)
x2=torch.randn(2,2)
x=pack_sequence([x1,x2], enforce_sorted=False)
y, (hn, cn) = rnn(x)
y,lens=pad_packed_sequence(y)
print the hidden state result:
print(torch.cat([hn[-2], hn[-1]], dim=1))
get result:
tensor([[-0.0265, -0.1149, -0.0466, 0.1080, 0.0901, 0.0865],
[ 0.0736, -0.2503, -0.0634, 0.0801, 0.1401, -0.0032]],
grad_fn=<CatBackward>)
print the output:
for i,j in enumerate(lens):
print(y[i][j-1])
get result:
tensor([-0.0265, -0.1149, -0.0466, 0.1080, 0.0901, 0.0865],
grad_fn=<SelectBackward>)
tensor([ 0.0736, -0.2503, -0.0634, 0.0932, 0.0962, -0.0915],
grad_fn=<SelectBackward>)
the second tensor is not the same with the hidden state!
Why?