What does pack_padded_sequence do?

nlp
1

Hi,

I have this code:

embedded = self.embedding(input)
packed = torch.nn.utils.rnn.pack_padded_sequence(embedded, input_lengths)
output, hidden = self.LSTM(packed, None)

Here, input has the shape = [20, 32] , embedded = [20, 32, 300] , input lengths = [32],

and packed.data.shape is [368, 300].

  1. What does it do? Why does it require input lengths?

  2. Why is the shape [368, 300]? Why should the LSTM accept this as input, LSTM should take [seq_length, batch_size, feature_size] right, how can then LSTM figure out the correct shape from the [368, 300] dimensional tensor? What attributes in the packed object allow this? The output.data has the shape [368, 512]? How does the LSTM process, since the output is per time step?

  3. What does the below operation do? It is clearly unpacking the packed input to LSTM, but how is it doing it?

output, _ = torch.nn.utils.rnn.pad_packed_sequence(output)

Any help would be great :slight_smile: