DistributedDataParallel() Hanfing

1

I have used DDP in my Transformer model, but when I execute, init_process_group is hanging.

command used: python -m torch.distributed.launch --nnodes=1 --node_rank=1 --nproc_per_node=1 --use_env standard.py

With the above command, my goal is to run my model on a single node with a single GPU.
the system has 8 GPUs, but I would like to use a single GPU, just to make DDP API is working

2

Was DDP working on this machine before and are you able to use e.g. all 8 GPUs or are all calls hanging?

3

@ptrblck
Issue was with launching processes, now resolved.

Thanks