Good to hear it was this issue!
Well, it might sound a bit weird, but it’s the default mechanism using multiprocessing.
In case you are lazily loading your data it’s not a big deal.
However, if you preload the data, this might cause an issue.
Generally I’m not sure, if you get a big performance improvement using multiple workers, if the data is already loaded.
That being said, you could use shared memory, which would do exactly as you’ve suggested.
I’ve created a small example yesterday for exactly this use case here. Maybe you could use it as well?