Understanding LSTM input

12

Hey @aatrey,

Thanks for the feedback!

I was wondering if there is a way to remove the last sequences once we hit the idx + self.seq_length condition instead of appending 0s as it does now.

I think there are several ways for solving this. First two that come to my mind:

  1. use Torch pad_packed_sequence for generating batches of sequences of the same length starting from sequences of variable length, and Torch pack_padded_sequence for recovering the sequences in their original form (you can think of these operations as the inverse of each other);
  2. since the root problem is that standard DataLoaders do not handle batches of variable size, write your own DataLoader with a custom collate_fn that handles variable size inputs (take a look here).

Hope these suggestions can be a good start for your troubleshooting, sorry if my answer couldn’t be more detailed but it has been quite a while since the last time I’ve worked with LSTM in PyTorch, and I don’t want to give you any bad advice.

Cheers :slight_smile: