What is the difference between DataParallel and DistributedDataParallel?

2

DataParallel is for performing training on multiple GPUs, single machine.
DistributedDataParallel is useful when you want to use multiple machines.

19 Likes