When should we use pack_sequence, pack_padded_sequence and pad_packed_sequence

Recently, I found pack_sequence, pack_padded_sequence, and pad_packed_sequence for RNN modules. But I am not sure when these functions are useful.

Q1. To use pack_padded_sequence, sorting tensors by length is needed for every mini-batch. This is cost. On the other hand, I heard pack_padded_sequence skip calculation of padded elements. Compared them, skipping calculation is much beneficial? The below link is an example of pack_padded_sequence.

Q2. Although pack_sequence looks similar to pack_sequence, pack_sequence doesn’t need to use padded data. But what kind of situations can we use this function?

Thank you!

With pack_sequence in PyTorch 0.4, you can directly call it without padding zeros.

>>> import torch
>>> import torch.nn.utils.rnn as rnn_utils
>>> a = torch.Tensor([1, 2, 3])
>>> b = torch.Tensor([4, 5])
>>> c = torch.Tensor([6])
>>> packed = rnn_utils.pack_sequence([a, b, c])

I’m having the same confusion as your Q2. The following link is the only use case about pack_sequence I found: Code.