Padding sequence in LSTM

nlp
2
  • If you use 0 for padding, then it shouldn’t be also used for any word in your vocabulary. Otherwise the model cannot distinguish between padding and the respective word.
  • pack_padded_sequence should take care of correctly backpropagating. That’s the overall purpose

Regarding the last 3 points in general: You might want to consider creating batches where all sequences within one batch have the same length. No need for padding or packing; see this older post. It’s simply convenient, and I use it all the time.