_MultiProcessingDataLoaderIter error in torch dataloader

Hi, while training a model with torch dataloader I am getting this error.

Exception ignored in: <function _MultiProcessingDataLoaderIter.__del__ at 0x7f3e3a495320>
Traceback (most recent call last):
  File "/home/coviamai/miniconda3/envs/aniket/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1324, in __del__
    self._shutdown_workers()
  File "/home/coviamai/miniconda3/envs/aniket/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1264, in _shutdown_workers
    if not self._shutdown:
AttributeError: '_MultiProcessingDataLoaderIter' object has no attribute '_shutdown'
Exception ignored in: <function _MultiProcessingDataLoaderIter.__del__ at 0x7f3e3a495320>
Traceback (most recent call last):
  File "/home/coviamai/miniconda3/envs/aniket/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1324, in __del__
    self._shutdown_workers()
  File "/home/coviamai/miniconda3/envs/aniket/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1264, in _shutdown_workers
    if not self._shutdown:
AttributeError: '_MultiProcessingDataLoaderIter' object has no attribute '_shutdown'

I’m not sure why self._shutdown is not recognized most likely in this line of code as it’s defined here. Are you running into a previous exception or does the stack trace show more errors?

I am getting the same exception quite regularly.

The stack trace was the same for me as in the original post. To try to get a better idea of what is going on, I inserted a breakpoint before if not self._shutdown if there was no _shutdown attribute. When I print out a stack trace at this breakpoint, however, it seems to vary randomly. For example, here are the last few lines of the stack trace the last two times I ran my code:

  /opt/homebrew/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py(928)wait()
-> with _WaitSelector() as selector:
  /opt/homebrew/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/selectors.py(349)__init__()
-> super().__init__()
  /Users/malcolm/venvs/ms_ml_utils2/lib/python3.9/site-packages/torch/utils/data/dataloader.py(1361)__del__()
-> self._shutdown_workers()
> /Users/malcolm/venvs/ms_ml_utils2/lib/python3.9/site-packages/torch/utils/data/dataloader.py(1292)_shutdown_workers()
-> if not self._shutdown:
  /opt/homebrew/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/queues.py(190)_start_thread()
-> self._close = Finalize(
  /opt/homebrew/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/util.py(199)__init__()
-> self._kwargs = kwargs or {}
  /Users/malcolm/venvs/ms_ml_utils2/lib/python3.9/site-packages/torch/utils/data/dataloader.py(1361)__del__()
-> self._shutdown_workers()
> /Users/malcolm/venvs/ms_ml_utils2/lib/python3.9/site-packages/torch/utils/data/dataloader.py(1292)_shutdown_workers()
-> if not self._shutdown:

Since _shutdown is defined in __init__() of the _MultiProcessingDataLoaderIter class, I tried inserting self._shutdown = False on the very first line of the init function, just to see what would happen. Doing this didn’t seem to make any difference—__init__ doesn’t seem to be invoked before the call to __del__.

I presume there is something not quite right about my dataloader but I’m having trouble working out what it is!