# What is the use of \`device\_ids\` in DDP constructor?

**URL:** <https://discuss.pytorch.org/t/what-is-the-use-of-device-ids-in-ddp-constructor/155116>\
**Category:** distributed\
**Created:** [June 26, 2022, 8:40pm UTC](https://discuss.pytorch.org/t/what-is-the-use-of-device-ids-in-ddp-constructor/155116 "2022-06-26T20:40:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Fei\_Liu](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fei_liu/32/36219_2.png) [@Fei\_Liu](https://discuss.pytorch.org/u/Fei_Liu)\
**Post date:** [June 26, 2022, 8:40pm UTC](https://discuss.pytorch.org/t/what-is-the-use-of-device-ids-in-ddp-constructor/155116/1 "2022-06-26T20:40:25Z")

</div>

I have a distributed setup with 1 gpu per node & four gpus/nodes in total. I noticed that it doesn’t matter whether I put `device_ids` and `output_device` in the DDP constructor, as long as I have the model device set up correctly (e.g. by `model.to(rank)`).

What’s the use of `device_ids` and `output_device`? Does setting them up help with speed or something else? I’m a bit confused and I didn’t find more detailed documents in the [DDP API page](https://pytorch.org/docs/stable/generated/torch.nn.parallel.DistributedDataParallel.html).

---

<div class="post-metadata">

**Author:** ![rvarm1](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rvarm1/32/15821_2.png) [@rvarm1](https://discuss.pytorch.org/u/rvarm1)\
**Post date:** [June 28, 2022, 1:21am UTC](https://discuss.pytorch.org/t/what-is-the-use-of-device-ids-in-ddp-constructor/155116/2 "2022-06-28T01:21:07Z")

</div>

If you’ve set up the model on the appropriate GPU for the rank, `device_ids` arg can be omitted, as the DDP doc mentions:

> Alternatively, `device_ids` can also be `None` .

Also, it actually appears `output_device` is unused by DDP now, possibly after some refactoring. This argument doesn’t have to be used and won’t have any effect, so it is fine to omit it as well.

---

<div class="post-metadata">

**Author:** ![Fei\_Liu](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fei_liu/32/36219_2.png) [@Fei\_Liu](https://discuss.pytorch.org/u/Fei_Liu)\
**Post date:** [June 28, 2022, 3:38am UTC](https://discuss.pytorch.org/t/what-is-the-use-of-device-ids-in-ddp-constructor/155116/3 "2022-06-28T03:38:55Z")

</div>

I’m still a bit confused. Say if I’m using singleGPU per node, then I’ll have to set up the model device correct first, otherwise it’ll complain with errors anyways. In other words, I need to do `DDP(model.to(rank), ...)` anyways, irrespective of adding `device_ids=[rank]` or not. Is it fair to say that the `device_ids` and `output_device` are just dummy (for now) and have no practical effects?