DataParallel and DistributedDataParallel stuck at 100% GPU usage

[rvarm1 (Rohan Varma)]

@FarisHijazi Does the same stuckness issue occur when trying training with the Gloo backend?