# Using convert\_sync\_batchnorm let my code be deadlock

**URL:** <https://discuss.pytorch.org/t/using-convert-sync-batchnorm-let-my-code-be-deadlock/95144>\
**Category:** distributed\
**Created:** [September 4, 2020, 6:51am UTC](https://discuss.pytorch.org/t/using-convert-sync-batchnorm-let-my-code-be-deadlock/95144 "2020-09-04T06:51:14Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mercer\_Alex](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mercer_alex/32/15290_2.png) [@mercer\_Alex](https://discuss.pytorch.org/u/mercer_Alex)\
**Post date:** [September 4, 2020, 6:51am UTC](https://discuss.pytorch.org/t/using-convert-sync-batchnorm-let-my-code-be-deadlock/95144/1 "2020-09-04T06:51:14Z")

</div>

when i use the code “net = torch.nn.SyncBatchNorm.convert\_sync\_batchnorm(net)” to replace BN with SyncBatchNorm, the code would be deadlock like this:

 ![image](https://discuss.pytorch.org/uploads/default/original/3X/3/5/358b80317d8478c933cee1c9d01cb23d8532eed2.png)

it seems to be a problem with dataloader. And the relevant code is as follows

 ![image](https://discuss.pytorch.org/uploads/default/original/3X/c/0/c072e4e054593df5765baa7da8b59ebffbe0eafb.png)

 ![image](https://discuss.pytorch.org/uploads/default/original/3X/8/d/8d04d004987395dd45ab135fd9024c5fd100d7f0.png)

Is there any kind person to help me？Thanks.

---

<div class="post-metadata">

**Author:** ![mrshenli](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mrshenli/32/12220_2.png) [@mrshenli](https://discuss.pytorch.org/u/mrshenli)\
**Post date:** [September 15, 2020, 3:22am UTC](https://discuss.pytorch.org/t/using-convert-sync-batchnorm-let-my-code-be-deadlock/95144/2 "2020-09-15T03:22:20Z")

</div>

The difference between `BatchNorm` and `SyncBatchNorm` is that `SyncBatchNorm` uses `torch.distributed.all_reduce` in the backward pass.

Two questions:

1. What args and env vars did you pass to `init_process_group`?
2. If you program, is there any other code that launches communication ops?