Crashing during using DataLoaders with several workers from different threads

In two projects I have the same problem.

Evnironemt: Linux, Ubunitu 16.04
Python: 3.9.1
Torch: 1.7.1

The problem lies in the case of using DataLoaders instantiated in the thread (main) for completely different datasets. After that data-loaders are used in different threads.

DataLoaderes are configured with default “num_workers = 4”
An attempt to have the ability from different threads to use different data loaders is crashing.

One workaround is to use “num_workers = 0” and it solves the problem…

It very likely has a connection with bug in Python: Issue 41567: multiprocessing.Pool from concurrent threads failure on 3.9.0rc1 - Python tracker

Stack traces:
File “/home/burlack/miniconda3/envs/pl/lib/python3.9/”, line 954, in _bootstrap_inner
File “/home/burlack/pl_new/pl_pytorch/pl_pytorch/utils/”, line 50, in run
retValues = f(self, *args)
File “/home/burlack/pl_new/pl_pytorch/pl_pytorch/utils/”, line 145, in local_training
metrics_meter = train_model(
File “/home/burlack/pl_new/pl_pytorch/pl_pytorch/utils/”, line 223, in train_model
for i, (data, label) in enumerate(train_loader):
File “/home/burlack/miniconda3/envs/pl/lib/python3.9/site-packages/torch/utils/data/”, line 352, in iter
return self._get_iterator()
File “/home/burlack/miniconda3/envs/pl/lib/python3.9/site-packages/torch/utils/data/”, line 294, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File “/home/burlack/miniconda3/envs/pl/lib/python3.9/site-packages/torch/utils/data/”, line 776, in init
self._worker_result_queue = multiprocessing_context.Queue() # type: ignore
File “/home/burlack/miniconda3/envs/pl/lib/python3.9/multiprocessing/”, line 103, in Queue
return Queue(maxsize, ctx=self.get_context())
File “/home/burlack/miniconda3/envs/pl/lib/python3.9/multiprocessing/”, line 40, in init
from .synchronize import SEM_VALUE_MAX as maxsize
ImportError: cannot import name ‘SEM_VALUE_MAX’ from partially initialized module ‘multiprocessing.synchronize’ (most likely due to a circular import) (/home/burlack/miniconda3/envs/pl/lib/python3.9/multiprocessing/
Exception ignored in: <function _MultiProcessingDataLoaderIter.del at 0x14c8389d61f0>
Traceback (most recent call last):
File “/home/burlack/miniconda3/envs/pl/lib/python3.9/site-packages/torch/utils/data/”, line 1203, in del
File “/home/burlack/miniconda3/envs/pl/lib/python3.9/site-packages/torch/utils/data/”, line 1150, in _shutdown_workers
if not self._shutdown:

1 Like

With python version 3.9.2, and torch version 3.8.1 the problem is still persist.

The import error points indeed towards a circular import. Would you mind creating an issue with the problem description and (if possible) a code snippet to reproduce the error?

Do you mean “Python 3.9.2 and PyTorch 1.8.1”?

Hello. Yes. It was a misprint.