# Module 'torch.distributed' has no attribute 'is\_initialized' **URL:** \ **Category:** distributed\ **Created:** [September 6, 2019, 8:56pm UTC](https://discuss.pytorch.org/t/module-torch-distributed-has-no-attribute-is-initialized/55391 "2019-09-06T20:56:27Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![zimmer550](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/bbe5ce/32.png) [@zimmer550](https://discuss.pytorch.org/u/zimmer550)\ **Post date:** [September 6, 2019, 8:56pm UTC](https://discuss.pytorch.org/t/module-torch-distributed-has-no-attribute-is-initialized/55391/1 "2019-09-06T20:56:27Z")
I am running inference using mmdetection ([https://github.com/open-mmlab/mmdetection](https://github.com/open-mmlab/mmdetection)) and I get the above error for this piece of code; ```auto model = init_detector("faster_rcnn_r50_fpn_1x.py", "faster_rcnn_r50_fpn_1x_20181010-3d1b3351.pth", device='cuda:0') img = 'intersection_unlabeled/frame0.jpg' result = inference_detector(model, img) show_result_pyplot(img, result, model.CLASSES) ``` And the full error log is: ```auto Traceback (most recent call last): File "C:/Users/sarim/PycharmProjects/thesis/pytorch_learning.py", line 355, in test() File "C:/Users/sarim/PycharmProjects/thesis/pytorch_learning.py", line 338, in test model = init_detector("faster_rcnn_r50_fpn_1x.py", "faster_rcnn_r50_fpn_1x_20181010-3d1b3351.pth", device='cuda:0') File "C:\Users\sarim\PycharmProjects\thesis\mmdetection\mmdet\apis\inference.py", line 36, in init_detector checkpoint = load_checkpoint(model, checkpoint) File "c:\users\sarim\appdata\local\programs\python\python37\lib\site-packages\mmcv\runner\checkpoint.py", line 188, in load_checkpoint load_state_dict(model, state_dict, strict, logger) File "c:\users\sarim\appdata\local\programs\python\python37\lib\site-packages\mmcv\runner\checkpoint.py", line 96, in load_state_dict rank, _ = get_dist_info() File "c:\users\sarim\appdata\local\programs\python\python37\lib\site-packages\mmcv\runner\utils.py", line 21, in get_dist_info initialized = dist.is_initialized() AttributeError: module 'torch.distributed' has no attribute 'is_initialized' ``` My pytorch version is 1.1.0 ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [September 6, 2019, 11:18pm UTC](https://discuss.pytorch.org/t/module-torch-distributed-has-no-attribute-is-initialized/55391/2 "2019-09-06T23:18:32Z")
Could you check your PyTorch version again (`print(torch. __version__ )`), as [`torch.distributed.is_initialized()`](https://pytorch.org/docs/1.1.0/distributed.html#torch.distributed.is_initialized) is in Pytorch 1.1.0. ---
**Author:** ![zimmer550](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/bbe5ce/32.png) [@zimmer550](https://discuss.pytorch.org/u/zimmer550)\ **Post date:** [September 9, 2019, 2:22am UTC](https://discuss.pytorch.org/t/module-torch-distributed-has-no-attribute-is-initialized/55391/3 "2019-09-09T02:22:15Z")
Of course, I did that before. Here is proof: ![cmd](https://discuss.pytorch.org/uploads/default/original/2X/6/6586617c104db6f9015ada023d6439b5c640c563.png) Anyways, I got it to work on windows by commenting out the lines that use distributed training. I read here that pytorch doesn’t support distributed training on windows but it does so on Linux: > > > \## ❓ Questions and Help > I installed according 'conda install pytorch-nightly-cp…u -c pytorch' (torch-nightly1.0.0.dev20181209), also I installed pytorch stable from official site following "conda install pytorch-cpu torchvision-cpu -c pytorch", but I still found this problem. > > > c:\\users\\xp\\test\\maskrcnn-benchmark\\maskrcnn\_benchmark\\utils\\comm.py in is\_main\_process() > 26 > 27 def is\_main\_process(): > \---\> 28 if not torch.distributed.is\_initialized(): > 29 return True > 30 return torch.distributed.get\_rank() == 0 > AttributeError: module 'torch.distributed' has no attribute 'is\_initialized' > > > In my win10 system, my torch is 1.0.0, and 'dir(torch.distributed)' returns > > \`\['\_\_builtins\_\_', '\_\_cached\_\_', '\_\_doc\_\_', '\_\_file\_\_', '\_\_loader\_\_', '\_\_name\_\_', '\_\_package\_\_', '\_\_path\_\_', '\_\_spec\_\_', 'deprecated', 'is\_available', 'torch'\]\` > > In my ubuntu system, I installed a GPU version, torch version is also 1.0.0, and 'dir(torch.distributed)' returns > > \`\['AllgatherOptions', 'AllreduceOptions', 'Backend', 'BarrierOptions', 'BroadcastOptions', 'FileStore', 'GatherOptions', 'GroupMember', 'PrefixStore', 'ProcessGroup', 'ProcessGroupGloo', 'ProcessGroupNCCL', 'ReduceOp', 'ReduceOptions', 'ScatterOptions', 'Store', 'TCPStore', 'Work', '\_\_builtins\_\_', '\_\_cached\_\_', '\_\_doc\_\_', '\_\_file\_\_', '\_\_loader\_\_', '\_\_name\_\_', '\_\_package\_\_', '\_\_path\_\_', '\_\_spec\_\_', '\_backend', '\_dist\_broadcast\_coalesced', '\_dist\_bucket\_tensors', '\_queue\_reduction', '\_sync\_params', '\_sync\_reduction', 'all\_gather', 'all\_gather\_multigpu', 'all\_reduce', 'all\_reduce\_multigpu', 'barrier', 'broadcast', 'broadcast\_multigpu', 'deprecated', 'destroy\_process\_group', 'dist\_backend', 'distributed\_c10d', 'gather', 'get\_backend', 'get\_rank', 'get\_world\_size', 'group', 'init\_process\_group', 'irecv', 'is\_available', 'is\_initialized', 'is\_mpi\_available', 'is\_nccl\_available', 'isend', 'new\_group', 'recv', 'reduce', 'reduce\_multigpu', 'reduce\_op', 'register\_rendezvous\_handler', 'rendezvous', 'scatter', 'send', 'string\_classes', 'timedelta', 'torch', 'warnings'\] > \` > > So why this happened, who can help? Many thanks. And this makes sense because I encountered no such issue when running the same code on Google Colab ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [September 9, 2019, 8:10am UTC](https://discuss.pytorch.org/t/module-torch-distributed-has-no-attribute-is-initialized/55391/4 "2019-09-09T08:10:44Z")
Thanks for checking and good to hear you’re figured it out. My second guess would also be that you’re using a platform which does not support distributed training. ---
**Author:** ![sytelus](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sytelus/32/4689_2.png) [@sytelus](https://discuss.pytorch.org/u/sytelus)\ **Post date:** [April 21, 2020, 6:22pm UTC](https://discuss.pytorch.org/t/module-torch-distributed-has-no-attribute-is-initialized/55391/5 "2020-04-21T18:22:06Z")
I wish `dist.is_initialized()` just returned always false instead of bombing out. This way the code is more cleaner between different platforms for non-distributed use. BTW, it seems same thing happens for methods like `is_gloo_available()` etc. ---
**Author:** ![mrshenli](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mrshenli/32/12220_2.png) [@mrshenli](https://discuss.pytorch.org/u/mrshenli)\ **Post date:** [April 21, 2020, 7:45pm UTC](https://discuss.pytorch.org/t/module-torch-distributed-has-no-attribute-is-initialized/55391/6 "2020-04-21T19:45:35Z")
There is `torch.distributed.is_available()` API to check if `distributed` package is available. APIs from `distributed` package is only available when `is_available` returns true. Let me add that to our docs. ---
**Author:** ![mrshenli](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mrshenli/32/12220_2.png) [@mrshenli](https://discuss.pytorch.org/u/mrshenli)\ **Post date:** [April 22, 2020, 2:45pm UTC](https://discuss.pytorch.org/t/module-torch-distributed-has-no-attribute-is-initialized/55391/7 "2020-04-22T14:45:01Z")
[https://github.com/pytorch/pytorch/pull/37021](https://github.com/pytorch/pytorch/pull/37021) is landed. The API doc of `torch.distributed.is_available` is added to master.