# Process group doesn't terminate by itself?

**URL:** <https://discuss.pytorch.org/t/process-group-doesnt-terminate-by-itself/118425>\
**Category:** distributed\
**Created:** [April 16, 2021, 10:52am UTC](https://discuss.pytorch.org/t/process-group-doesnt-terminate-by-itself/118425 "2021-04-16T10:52:16Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wenhao-Yang](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wenhao-yang/32/37202_2.png) [@Wenhao-Yang](https://discuss.pytorch.org/u/Wenhao-Yang)\
**Post date:** [April 16, 2021, 10:52am UTC](https://discuss.pytorch.org/t/process-group-doesnt-terminate-by-itself/118425/1 "2021-04-16T10:52:16Z")

</div>

Has anyone encountered a similar problem?

**The Python script could not exit by itself without Ctrl+C.** I try to train a model on 2 GPU in a ubuntu server.

Training and evaluation are fine. But after “main()” finishing, the python process won’t terminate by itself. I checked the gpu( `nvidia-smi` ) and cpu usage ( `htop` ). The process still occupied those resources, which means the process group could not be kill by itself. And if I training with only one GPU, it’s fine.

Here’s my opinions:

- `group_process` could not terminated for some reason? Except for initializaiton by `torch.distributed.init_process_group()`, I didn’t find any solutions in torch Documentation to termanate the group.
- Deleting variable like **model** could not help it out.
- Is this a bug for torch or python ?
- Did I make mistakes in using multi-gpu?

---

<div class="post-metadata">

**Author:** ![Yanli\_Zhao](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yanli_zhao/32/16018_2.png) [@Yanli\_Zhao](https://discuss.pytorch.org/u/Yanli_Zhao)\
**Post date:** [April 16, 2021, 5:24pm UTC](https://discuss.pytorch.org/t/process-group-doesnt-terminate-by-itself/118425/2 "2021-04-16T17:24:40Z")

</div>

code snippets?

also, why not use init\_process\_group()?

---

<div class="post-metadata">

**Author:** ![BramVanroy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bramvanroy/32/28628_2.png) [@BramVanroy](https://discuss.pytorch.org/u/BramVanroy)\
**Post date:** [April 16, 2021, 5:49pm UTC](https://discuss.pytorch.org/t/process-group-doesnt-terminate-by-itself/118425/3 "2021-04-16T17:49:39Z")

</div>

From what I remember you have to destroy the distributed groups, so at the end of your code:

```auto
import torch.distributed as dist
dist.destroy_process_group()

```

---

<div class="post-metadata">

**Author:** ![Wenhao-Yang](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wenhao-yang/32/37202_2.png) [@Wenhao-Yang](https://discuss.pytorch.org/u/Wenhao-Yang)\
**Post date:** [April 17, 2021, 1:56am UTC](https://discuss.pytorch.org/t/process-group-doesnt-terminate-by-itself/118425/4 "2021-04-17T01:56:10Z")

</div>

**Thanks for your advice! 😬**  
I tried `destroy_process_group()` but it didn’t work.

In the end, I fixed it by deleting the following Potential conflict line:  
`torch.multiprocessing.set_sharing_strategy('file_system')`

It seems that this line is compatible with

```auto
torch.distributed.init_process_group(backend="nccl", init_method='file:///home/xxx//sharedfile', rank=0, world_size=1)

```

instead of

```auto
torch.distributed.init_process_group(backend="nccl", init_method='tcp://localhost:32546', rank=0, world_size=1)

```

and it may cause the problems.

---

<div class="post-metadata">

**Author:** ![Wenhao-Yang](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wenhao-yang/32/37202_2.png) [@Wenhao-Yang](https://discuss.pytorch.org/u/Wenhao-Yang)\
**Post date:** [April 17, 2021, 2:13am UTC](https://discuss.pytorch.org/t/process-group-doesnt-terminate-by-itself/118425/5 "2021-04-17T02:13:36Z")

</div>

Sorry for missing code snippets. This is the first time to create a topic in the community. Here are my codes. And I have already find the potential conflict line:

```auto
# import ...
os.environ['CUDA_VISIBLE_DEVICES'] = “0,1”

def main():
    train_loader = torch.utils.data.DataLoader(train_dir,
                                                                       batch_size=args.batch_size, 
                                                                       shuffle=args.shuffle,
                                                                        **kwargs)
    
    # --------------------------------------------------- confilct line -------------------------------------------
    torch.distributed.init_process_group(backend="nccl",
                                                              init_method='tcp://localhost:32546',
                                                              rank=0, 
                                                              world_size=1)

    # ----------------------------------- line without the problem ---------------------------------------
    torch.distributed.init_process_group(backend="nccl", init_method='file:///home/xxx/sharedfile',
                                                              rank=0,
                                                              world_size=1)

    model = DistributedDataParallel(model.cuda())

    # train()...
    # eval()...

if __name__ == ' __main__':
    main()

```

Thanks, anyway! 😬