Hi all,
I have a PR to fix a bug when DDP gradient accumulation is used with static_graph=True and no_sync. Can I get anyone to take a look at [DDP] Fix static_graph=True + no_sync() gradient accumulation regression by kasper0406 · Pull Request #187732 · pytorch/pytorch · GitHub ?
I hit the bug executing some pytorch code on my workstation.