Perhaps my replies are not applicable. DistributedDataParallel seems much more complicated :-). Please ignore.