# More gpu accelerate the speed of dataloader?

**URL:** <https://discuss.pytorch.org/t/more-gpu-accelerate-the-speed-of-dataloader/82962>\
**Category:** Uncategorized\
**Created:** [May 26, 2020, 1:53pm UTC](https://discuss.pytorch.org/t/more-gpu-accelerate-the-speed-of-dataloader/82962 "2020-05-26T13:53:00Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![XavierLinNow](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/x/e274bd/32.png) [@XavierLinNow](https://discuss.pytorch.org/u/XavierLinNow)\
**Post date:** [May 26, 2020, 1:53pm UTC](https://discuss.pytorch.org/t/more-gpu-accelerate-the-speed-of-dataloader/82962/1 "2020-05-26T13:53:00Z")

</div>

In my experiment, I test different num of gpus and num\_wokers in dataloader. Here is the result

| num gpu | batch size | num worker | one batch time(get data and forward backward) | one batch time(get data) |
| --- | --- | --- | --- | --- |
| 2 | 512 | 128 | 0.8 | 0.3 |
| 2 | 512 | 32 | 0.8 | 0.3 |
| 2 | 512 | 16 | 0.8 | 0.3 |
| 2 | 512 | 8 | 0.8 | 0.3 |
| 2 | 512 | 2 | 0.8 | 0.3 |
| 2 | 512 | 1 | 3.7 | 3.2 |
| 8 | 512 | 32 | 0.3 | 0.08 |
| 8 | 1024 | 32 | 0.55 | 0.14 |

There are two weird thing. 1) is more gpu accelerate the speedup of dataloader. 2) when increase the num\_worker, it doesn’t accelerate the speed of getting data.

---

<div class="post-metadata">

**Author:** ![SANTOSH\_S](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/santosh_s/32/24798_2.png) [@SANTOSH\_S](https://discuss.pytorch.org/u/SANTOSH_S)\
**Post date:** [May 26, 2020, 6:23pm UTC](https://discuss.pytorch.org/t/more-gpu-accelerate-the-speed-of-dataloader/82962/2 "2020-05-26T18:23:55Z")

</div>

Interesting. Are these experiments conducted with `pin_memory=True` in the DataLoaders ?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [May 27, 2020, 8:38am UTC](https://discuss.pytorch.org/t/more-gpu-accelerate-the-speed-of-dataloader/82962/3 "2020-05-27T08:38:22Z")

</div>

Could you share the code you’ve used to measure the data loading speed?  
Note that too many workers might degrade the performance as explained in [this post](https://discuss.pytorch.org/t/how-to-prefetch-data-when-processing-with-gpu/548/19).