Torch.distributed.reduce why need to reduce all tensors

Hi everyone, why we need to reduce all tensors across all machines? What’s the reason