My system is Windows, and when num_work is greater than 0 (my tests are 4 and 8), I will always call the py file of the training code

1

This is an interesting issue that I discovered when using yolov10. When my num_work is configured to 0, the training code executes normally. However, when the num_work parameter is 4/8, when my code reaches init under InfiniteDataLoader, self. iterator=super() is called__ When using the item__ () method, I immediately create a process with a number of num_works to call the py file I used to train the code. This repeated process ultimately leads to system OOM. I really want to record a video of this interesting issue, but unfortunately, I don’t know how to post it. I have to take screenshots of the important part of the log code and send them out

I have heard before that it is best not to set num_work to a value greater than 0 in Windows. Now, this question makes me curious. Why can’t this parameter be set to greater than 0 in Windows? From a bottom-up design perspective

this is when num_work was zeros:

1719241711128
17192417111281273×505 112 KB

this is when num_work was 8 or 4

1719241884098
17192418840981281×960 170 KB

2

From the Windows FAQ:

The implementation of multiprocessing is different on Windows, which uses spawn instead of fork. So we have to wrap the code with an if-clause to protect the code from executing multiple times.

Add the if-clause protection to avoid this behavior.

3

But I am still very interested in this bug, why does it appear on Windows repeatedly calling the main process? Don’t you know how to do it on Ubuntu?

4

The quoted docs explain that the spawn method is used by default in Windows and you can try to set it explicitly on Linux to recreate the mentioned issues.