# What is the \`\_replicated\_tensor\_module\` in DDP?

**URL:** <https://discuss.pytorch.org/t/what-is-the-replicated-tensor-module-in-ddp/183035>\
**Category:** distributed\
**Created:** [June 28, 2023, 3:28am UTC](https://discuss.pytorch.org/t/what-is-the-replicated-tensor-module-in-ddp/183035 "2023-06-28T03:28:07Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Fei\_Liu](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fei_liu/32/36219_2.png) [@Fei\_Liu](https://discuss.pytorch.org/u/Fei_Liu)\
**Post date:** [June 28, 2023, 3:28am UTC](https://discuss.pytorch.org/t/what-is-the-replicated-tensor-module-in-ddp/183035/1 "2023-06-28T03:28:07Z")

</div>

I’m looking at the DDP source code `train()` function ([reference](https://pytorch.org/docs/stable/_modules/torch/nn/parallel/distributed.html#DistributedDataParallel)):

```auto
    def train(self, mode=True):
        super().train(mode)
        if self._use_replicated_tensor_module:
            self._replicated_tensor_module.train(mode) # type: ignore[union-attr]
        return self

```

What is the `_replicated_tensor_module` for?

Context: I’m trying to understand the difference between `ddp_model.train()` and `ddp_model.module.train()`.