Hello there.
Do we have a quick way to disable the overlapping of backward computation and all-reduce communication like delay_allreduce argument of apex.parallel?
Or do we have any easy way to force everything into a single one GPU stream?
Thanks a lot