Torch.distributed.reduce why need to reduce all tensors

[Xiaoyu_Song]

Hi everyone, why we need to reduce all tensors across all machines? What’s the reason