DataLoader with num_workers > 0 raising error with mongodb

Hello everyone,
I have the following error:

    for batch_idx, (data, target, idx, kws_target) in enumerate(train_loader):
  File "..\Python\Python37\lib\site-packages\torch\utils\data\", line 291, in __iter__
    return _MultiProcessingDataLoaderIter(self)
  File "..\Python\Python37\lib\site-packages\torch\utils\data\", line 737, in __init__
  File "..multiprocessing\", line 112, in start 
    self._popen = self._Popen(self)
  File "..\Python\Python37\lib\multiprocessing\", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "..\Python\Python37\lib\multiprocessing\", line 322, in _Popen
    return Popen(process_obj)
  File "..\Python\Python37\lib\multiprocessing\", line 89, in __init__
    reduction.dump(process_obj, to_child)
  File "..\Python\Python37\lib\multiprocessing\", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects

when I’m iterating over dataset.
In the getitem function I’m using mongoDB client “find” function .

When I initialize the data loader with num_workers=0, everything runs just fine.

What causes this error? And how can I solve it?

This seems to be a multiprocessing issue. Could you try to use the spawn method and rerun your code?