Hi, there I am trying to train a model and getting a strange error…
If I run the model on a small subsection of my data without the data loader argument ‘num_workers’ the model runs fine…
However, if I attempt to run the model on the whole dataset (or even a significant amount of the dataset) the model hits a bottleneck - still without ‘num_workers’.
I figured the simple solution to this error is to set ‘num_workers’ - I have tried a number of different workers up to 32. This is where I get my error as seen below…
In summary, if I want to train my model on the whole dataset I am stuck as the model either bottlenecks when ‘num_workers’ is removed from the data loader or I get this error if I set it to anything else.
