I have this same issue with trying to run DDP with a model with ComplexFloat parameters: NCCL Backend does not support ComplexFloat data type · Issue #71613 · pytorch/pytorch · GitHub
In passing in the GH issue, a workaround was suggested to view the complex as real before passing to NCCL. But I don’t quite follow. I have tried adding register_backward_hook, and converting the gradients there using torch.view_as_real, but that doesn’t seem to do anything. I also tried adding register_comm_hook’s, which seemed promising, but it appears they can only be registered on the DDP object, which in this issue doesn’t exist because it throws the error in the DDP call.
It appears this will be fixed, but can someone point to how to modify the example in the issue to successfully use DDP with ComplexFloat models?