DistributedDataParallel() for Transformer model

[gnadaf]

Done anyone know how to add DistributedDataParallel() for the transformer model?

Transformer model: https://pytorch.org/tutorials/beginner/transformer_tutorial.html