I’m training one model on two different sources of data.
My training loop is essentially
train one batch from source A
train one batch from source B
Note that the code to train from the two sources isnt exactly the same, but the model is.
When I initialize both DataLoaders with num_workers, I get the following execption:
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the ‘spawn’ start method
Using the spawn method causes other side effects, so I would prefer to stay in this mode.
It there a way to get this to work?