How pytorch's parallel method and distributed method works?

@rasbt

Any views on this thread
How to parallelise a pytorch model on GPU?