What are the (dis) advantages of persistent_workers

If you’re using num_workers=0, there are no worker processes, so the persistent worker flag will have no effect at all :slight_smile:
But indeed, if your dataset is completely in memory and you don’t have heavy preprocessing, then you can use 0 workers and it will run just fine.

3 Likes