Using `multiprocessing' in PyTorch on Windows10 got error-RuntimeError: Couldn't open shared file mapping: <torch_13684_4004974554>, error code: <0>

windows
1

Hi,

I am currently running a PyTorch code on Windows10 using PyCharm. This code firstly utilised DataLoader' function (num_workers’=4) to load training data:

train_loader = DataLoader(train_dset, batch_size, shuffle=True,
                                  num_workers=4, collate_fn=trim_collate)

Then, in training process, it utilised a `for’ loop to load training data and train the model:

for i, (v, norm_bb, q, target, _, _, bb, spa_adj_matrix,
                        sem_adj_matrix) in enumerate(train_loader):

Error: I got the following error messages when running above `for’ loop:

0%|          | 0/6934 [00:00<?, ?it/s]Traceback (most recent call last):
  File "E:\PyTorch_env\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
    reduction.dump(process_obj, to_child)
  File "E:\PyTorch_env\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "E:\PyTorch_env\lib\site-packages\torch\multiprocessing\reductions.py", line 286, in reduce_storage
    metadata = storage._share_filename_()
RuntimeError: Couldn't open shared file mapping: <torch_13684_4004974554>, error code: <0>
python-BaseException
Traceback (most recent call last):
  File "E:\PyTorch_env\lib\multiprocessing\spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
python-BaseException
  0%|          | 0/6934 [00:07<?, ?it/s]

It seems that there are some issues with the `multiprocessing’ function on Windows system. Could you please let me know if there are any possible solutions for this?

The environment settings:

  1. Windows10, PyCharm
  2. PyTorch v1.0.1, torchvision v0.2.2, Python 3.7.11
  3. One GPU node

Many thanks!

3

For the sake of completeness: issue is also tracked here.

4

Many thanks, @ptrblck.