Module 'torch.distributed' has no attribute 'is_initialized'

distributed
1

I am running inference using mmdetection (https://github.com/open-mmlab/mmdetection) and I get the above error for this piece of code;

    model = init_detector("faster_rcnn_r50_fpn_1x.py", "faster_rcnn_r50_fpn_1x_20181010-3d1b3351.pth", device='cuda:0')
    img = 'intersection_unlabeled/frame0.jpg'
    result = inference_detector(model, img)
    show_result_pyplot(img, result, model.CLASSES)

And the full error log is:

Traceback (most recent call last):
  File "C:/Users/sarim/PycharmProjects/thesis/pytorch_learning.py", line 355, in <module>
    test()
  File "C:/Users/sarim/PycharmProjects/thesis/pytorch_learning.py", line 338, in test
    model = init_detector("faster_rcnn_r50_fpn_1x.py", "faster_rcnn_r50_fpn_1x_20181010-3d1b3351.pth", device='cuda:0')
  File "C:\Users\sarim\PycharmProjects\thesis\mmdetection\mmdet\apis\inference.py", line 36, in init_detector
    checkpoint = load_checkpoint(model, checkpoint)
  File "c:\users\sarim\appdata\local\programs\python\python37\lib\site-packages\mmcv\runner\checkpoint.py", line 188, in load_checkpoint
    load_state_dict(model, state_dict, strict, logger)
  File "c:\users\sarim\appdata\local\programs\python\python37\lib\site-packages\mmcv\runner\checkpoint.py", line 96, in load_state_dict
    rank, _ = get_dist_info()
  File "c:\users\sarim\appdata\local\programs\python\python37\lib\site-packages\mmcv\runner\utils.py", line 21, in get_dist_info
    initialized = dist.is_initialized()
AttributeError: module 'torch.distributed' has no attribute 'is_initialized'

My pytorch version is 1.1.0

2

Could you check your PyTorch version again (print(torch.__version__)), as torch.distributed.is_initialized() is in Pytorch 1.1.0.

3

Of course, I did that before. Here is proof:

cmd
cmd1103Ă—646 21 KB

Anyways, I got it to work on windows by commenting out the lines that use distributed training. I read here that pytorch doesn’t support distributed training on windows but it does so on Linux:

And this makes sense because I encountered no such issue when running the same code on Google Colab

1 Like
4

Thanks for checking and good to hear you’re figured it out. My second guess would also be that you’re using a platform which does not support distributed training.

1 Like
5

I wish dist.is_initialized() just returned always false instead of bombing out. This way the code is more cleaner between different platforms for non-distributed use. BTW, it seems same thing happens for methods like is_gloo_available() etc.

6

There is torch.distributed.is_available() API to check if distributed package is available. APIs from distributed package is only available when is_available returns true. Let me add that to our docs.

7

https://github.com/pytorch/pytorch/pull/37021 is landed. The API doc of torch.distributed.is_available is added to master.