I was reading on multiprocessing and it seems like lots of problems are actually related to how bad Python’s multiprocessing is in general (worse in Python2).
Instead of trying to find all the quirks and workarounds for deadlocks (killed workers are tricky) PyTorch could switch to loky. It is an almost drop-in replacement for multiprocessing, but with consistent spawn behavior, and works fine with Python2.
What I like the most is to get an exception when a worker dies, instead of a hard crash, or worse a deadlock.