# Sharing dataset between subprocess when using DistributedDataParallel

**URL:** <https://discuss.pytorch.org/t/sharing-dataset-between-subprocess-when-using-distributeddataparallel/110997>\
**Category:** distributed\
**Created:** [February 5, 2021, 6:03am UTC](https://discuss.pytorch.org/t/sharing-dataset-between-subprocess-when-using-distributeddataparallel/110997 "2021-02-05T06:03:40Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![osalpekar](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/o/5f9b8f/32.png) [@osalpekar](https://discuss.pytorch.org/u/osalpekar)\
**Post date:** [February 5, 2021, 7:00pm UTC](https://discuss.pytorch.org/t/sharing-dataset-between-subprocess-when-using-distributeddataparallel/110997/2 "2021-02-05T19:00:14Z")

</div>

Can you use PyTorch DataLoader? If you implement the ` __getitem__ ` function, the batches will be lazily read into memory. Each DDP replica will then have one DataLoader, and each DataLoader will load the data lazily, so there shouldn’t be as much memory pressure.

Relevant Forums Post: [How to use dataset larger than memory?](https://discuss.pytorch.org/t/how-to-use-dataset-larger-than-memory/37785)

---

_[View the full topic](https://discuss.pytorch.org/t/sharing-dataset-between-subprocess-when-using-distributeddataparallel/110997)._