# How to solve "RuntimeError: Address already in use" in pytorch distributed training?

**URL:** <https://discuss.pytorch.org/t/how-to-solve-runtimeerror-address-already-in-use-in-pytorch-distributed-training/39278>\
**Category:** distributed\
**Created:** [March 8, 2019, 8:46am UTC](https://discuss.pytorch.org/t/how-to-solve-runtimeerror-address-already-in-use-in-pytorch-distributed-training/39278 "2019-03-08T08:46:08Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![keira\_feng](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/cab0a1/32.png) [@keira\_feng](https://discuss.pytorch.org/u/keira_feng)\
**Post date:** [March 8, 2019, 8:46am UTC](https://discuss.pytorch.org/t/how-to-solve-runtimeerror-address-already-in-use-in-pytorch-distributed-training/39278/1 "2019-03-08T08:46:08Z")

</div>

In the pytorch distributed training, I met a RuntimeError as following:

```auto
Traceback (most recent call last):
  File "visual/distribution_train.py", line 387, in <module>
    main()
  File "visual/distribution_train.py", line 67, in main
    spawn(main_worker, nprocs=ngpus_per_node, args=(ngpus_per_node, args))
  File "/public/home/fengm/.conda/envs/fm_pytorch_env/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 167, in spawn
    while not spawn_context.join():
  File "/public/home/fengm/.conda/envs/fm_pytorch_env/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 114, in join
    raise Exception(msg)
Exception:

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/public/home/fengm/.conda/envs/fm_pytorch_env/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
    fn(i, *args)
  File "/public/home/fengm/vehicle_reid/pytorch-pose-master/visual/distribution_train.py", line 74, in main_worker
    dist.init_process_group(backend=args.dist_backend, init_method=args.dist_url,world_size=args.world_size, rank=args.rank)
  File "/public/home/fengm/.conda/envs/fm_pytorch_env/lib/python3.6/site-packages/torch/distributed/distributed_c10d.py", line 354, in init_process_group
    store, rank, world_size = next(rendezvous(url))
  File "/public/home/fengm/.conda/envs/fm_pytorch_env/lib/python3.6/site-packages/torch/distributed/rendezvous.py", line 95, in _tcp_rendezvous_handler
    store = TCPStore(result.hostname, result.port, world_size, start_daemon)
RuntimeError: Address already in use

```

pytorch distributed initial setting is

```auto
torch.multiprocessing.spawn(main_worker, nprocs=8, args=(8, args))
torch.distributed.init_process_group(backend='nccl', init_method='tcp://110.2.1.101:8900',world_size=4, rank=0)

```

There are 10 nodes with gpu mounted under the master node. The master node doesn’t have GPU. I used the **slurm system** to submit my task and my task is randomly assigned to worker node. ‘110.2.1.101’ in init\_method is the master IP. I don’t kown whether is the init\_method wrong?  
Is there anyone have met it before ? Who can help me to fix this bug?

---

<div class="post-metadata">

**Author:** ![pietern](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pietern/32/4981_2.png) [@pietern](https://discuss.pytorch.org/u/pietern)\
**Post date:** [March 12, 2019, 5:12pm UTC](https://discuss.pytorch.org/t/how-to-solve-runtimeerror-address-already-in-use-in-pytorch-distributed-training/39278/2 "2019-03-12T17:12:57Z")

</div>

What do you run in `main_worker` and where do the `world_size=4` and `rank=0` arguments to `init_process_group` come from? Are they hard coded, or do you list a single example?

The error itself means that multiple processes try to bind to the address and port, so I assume you are trying to run multiple processes with `rank=0`.

---

<div class="post-metadata">

**Author:** ![HaFred](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hafred/32/27581_2.png) [@HaFred](https://discuss.pytorch.org/u/HaFred)\
**Post date:** [February 23, 2021, 4:25am UTC](https://discuss.pytorch.org/t/how-to-solve-runtimeerror-address-already-in-use-in-pytorch-distributed-training/39278/3 "2021-02-23T04:25:59Z")

</div>

Sorry to bring this up. I ran into the same issues, I wonder did you solve it?

---

<div class="post-metadata">

**Author:** ![Yanli\_Zhao](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yanli_zhao/32/16018_2.png) [@Yanli\_Zhao](https://discuss.pytorch.org/u/Yanli_Zhao)\
**Post date:** [February 23, 2021, 7:56pm UTC](https://discuss.pytorch.org/t/how-to-solve-runtimeerror-address-already-in-use-in-pytorch-distributed-training/39278/4 "2021-02-23T19:56:11Z")

</div>

you can set a free port

---

<div class="post-metadata">

**Author:** ![M\_Saiful\_Bari](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/m_saiful_bari/32/37563_2.png) [@M\_Saiful\_Bari](https://discuss.pytorch.org/u/M_Saiful_Bari)\
**Post date:** [April 25, 2021, 5:18pm UTC](https://discuss.pytorch.org/t/how-to-solve-runtimeerror-address-already-in-use-in-pytorch-distributed-training/39278/5 "2021-04-25T17:18:09Z")

</div>

Faced the same issue.

My Solution: It simply means that the GPU is already occupied under some other ddp training. Try deleting all the processes related to the running GPU and run the process again.