# Multiple forward functions in DP and DDP

**URL:** <https://discuss.pytorch.org/t/multiple-forward-functions-in-dp-and-ddp/135029>\
**Category:** distributed\
**Created:** [October 25, 2021, 9:42am UTC](https://discuss.pytorch.org/t/multiple-forward-functions-in-dp-and-ddp/135029 "2021-10-25T09:42:24Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![seungjun](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/90ced4/32.png) [@seungjun](https://discuss.pytorch.org/u/seungjun)\
**Post date:** [October 27, 2021, 1:13pm UTC](https://discuss.pytorch.org/t/multiple-forward-functions-in-dp-and-ddp/135029/5 "2021-10-27T13:13:47Z")

</div>

Hi @mrshenli,

Thank you for confirming the 1st option and pointing to the related part of the DDP source code.  
I checked the DDP implementation and it seems that option 1 is the only possible way for now.

`forward` is the only function that DDP supports safe parallelization and going for option 3 would be an adventure.

By the way, I’m not sure if I could avoid the function call patterns like `forward`, `forward`, `backward` you mentioned.  
Thank you very much and I will post here when I come up with a nice solution.

Best,  
Seungjun

---

_[View the full topic](https://discuss.pytorch.org/t/multiple-forward-functions-in-dp-and-ddp/135029)._