Issue with multiprocessing semaphore tracking

4

I run to the same issue with the warning (when running some preprocessing on GPU), except for now, I didn’t see a slow down in the running time of the code, yet.

python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))

The message seems harmful, but pretty annoying since it is displayed whenever the multiprocessing takes place.

One way to hide this warning is to ignore it using the Python module warnings. Also, redirecting the stderr may be temporary solution.

However, warnings.filterwarnings() DOES not work for this particular warning (and probably for all the warnings of the Python module multiprocessing.semaphor_tracker(). I don’t really understand why. I tried different ways using regular expressions, but nothing seem to work.

Things that work:

  1. python -W ignore your_script.py --> ignore ALL the warnings within your script. (not recommended).
  2. python -W ignore:semaphore_tracker:UserWarning your_script.py or python -W 'ignore:semaphore_tracker:UserWarning' your_script.py seem to ignore exactly the above warning. It can also done by using the environment variable PYTHONWARNINGS:
export PYTHONWARNINGS='ignore:semaphore_tracker:UserWarning'

python your_script.py

Python 3.7.1, Pytorch 1.0.0.

1 Like