If you’re using num_workers=0
, there are no worker processes, so the persistent worker flag will have no effect at all
But indeed, if your dataset is completely in memory and you don’t have heavy preprocessing, then you can use 0 workers and it will run just fine.
3 Likes