What's the difference betweem these two DataParallel methods?

1

Hey, I am considering to run my model on multiple GPU in one machine. And I found these two DataParallel methods in Pytorch Documentation.

  1. https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html
    Seems easier but somewhat vague.

  2. https://pytorch.org/tutorials/beginner/former_torchies/parallelism_tutorial.html
    Seems more specific

What’s the difference between these two pipelines? Which should I choose?

(BTW, I am training a Generative Adversarial Network. Are there some fantastic public Multi-GPUs codes I can refer to?

Thanks in advance~

2

I would use the first pipeline if I don’t need the precise control offered by the second pipeline. Besides, using the second pipeline means modifying the code of your model. I personally prefer the model code fixed and separated from training settings (to me using Multi-GPU is more of a training config rather than a model design).

Best

1 Like