GRU and padded sequences | Tipps and Tricks

[vdw (Chris)]

@Unity05

My first idea was to sort the sequences by their length, so that the total length difference in a padded batch would be as small as possible. Now I’m wondering whether there is an even better way to prevent the sequence padding to falsifie the networks output (by too much).

The BucketIterator creates batches with sequences of the same or similar lengths. Or you can create your own Sampler to create batches with sequences of the same length; see this post as an example.

The comments from @googlebot are of course valid, but it’s generally a good idea to avoid batches where the lengths of the sequences vary a great deal.