Is there issue or performance difference between single gpu with ddp and single gpu in single process?
@Kevin_Cha There shouldn’t be significant overhead, but what is the reason to use ddp with a single gpu? Usually it makes sense to use ddp when training with multiple GPUs.