# Write DataLoader's collate\_fn with pad\_packed\_sequence for LSTMs

**URL:** <https://discuss.pytorch.org/t/write-dataloaders-collate-fn-with-pad-packed-sequence-for-lstms/20892>\
**Category:** Uncategorized\
**Created:** [July 9, 2018, 8:14am UTC](https://discuss.pytorch.org/t/write-dataloaders-collate-fn-with-pad-packed-sequence-for-lstms/20892 "2018-07-09T08:14:16Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Maghoumi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/maghoumi/32/15135_2.png) [@Maghoumi](https://discuss.pytorch.org/u/Maghoumi)\
**Post date:** [July 9, 2018, 8:14am UTC](https://discuss.pytorch.org/t/write-dataloaders-collate-fn-with-pad-packed-sequence-for-lstms/20892/1 "2018-07-09T08:14:16Z")

</div>

I’m very new to PyTorch and my problem involves LSTMs with inputs of variable sizes.

Because each training example has a different size, what I’m trying to do is to write a custom `collate_fn` to use with DataLoader to create mini-batches of my data. To my understanding, I’d need to implement my own `collate_fn` and use `pad_packed_sequence` somehow…

I have tried looking at examples online, but nobody seems to be doing it like me ☹  
The problem arises from the fact that `collate_fn` gets passed the input `batch` which is a list of tuples of `(training_tensor, label)` and I can’t seem to figure out how to properly convert it to a suitable datatype and pass the tensors to `pad_packed_sequence`.

What is the correct way of doing this?  
A more general question: is there something better than `DataLoader` that works nicely with LSTMs?

---

<div class="post-metadata">

**Author:** ![Maghoumi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/maghoumi/32/15135_2.png) [@Maghoumi](https://discuss.pytorch.org/u/Maghoumi)\
**Post date:** [November 6, 2018, 9:59pm UTC](https://discuss.pytorch.org/t/write-dataloaders-collate-fn-with-pad-packed-sequence-for-lstms/20892/2 "2018-11-06T21:59:23Z")

</div>

I figured this out myself a while back. I posted a detailed solution here:

[https://www.codefull.org/2018/11/use-pytorchs-dataloader-with-variable-length-sequences-for-lstm-gru/](https://www.codefull.org/2018/11/use-pytorchs-dataloader-with-variable-length-sequences-for-lstm-gru/)

---

<div class="post-metadata">

**Author:** ![valentinoli](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/valentinoli/32/45431_2.png) [@valentinoli](https://discuss.pytorch.org/u/valentinoli)\
**Post date:** [January 3, 2022, 8:51am UTC](https://discuss.pytorch.org/t/write-dataloaders-collate-fn-with-pad-packed-sequence-for-lstms/20892/3 "2022-01-03T08:51:35Z")

</div>

@Maghoumi thanks for your post! Someone also posted this here later in 2019 in case it might be useful to someone:

> **[Pad pack sequences for Pytorch batch processing with DataLoader](https://suzyahyah.github.io/pytorch/2019/07/01/DataLoader-Pad-Pack-Sequence.html)**
>
> Pytorch setup for batch sentence/sequence processing - minimal working example. The pipeline consists of the following:

Also note that a boolean parameter `enforce_sorted` was added to `torch.nn.utils.rnn.pack_padded_sequence()` in December 2018 (see [this issue](https://github.com/pytorch/pytorch/issues/3584)). So there is no need anymore to provide a sorted sequence, it is sorted internally when `enforce_sorted=False`.