DistributedDataParallel does not reduce the training time

Yes, you are right. Even I don’t set num_replicas, it is set to be the number of process automatically. Thank you.