Does running multiple copies of a model (like resnet18) on the same GPU have any benefits like parallel execution? I would like to find the loss/accuracy on two different datasets and I was wondering if it can done more efficiently on a single GPU. I am currently evaluating the datasets sequentially.
Thanks!
Most likely you won’t see a performance benefit, as a single ResNet might already use all GPU resources, so that an overlapping execution wouldn’t be possible.
If you want to try it nevertheless, you could have a look at the CUDA streams docs. Note however, that you would be responsible for proper synchronizations to prevent race conditions etc.
1 Like
If I intend to train multiple small models like LeNet in parallel, do I have to create n CUDA streams with each model training function in its own with block?
Can you please give pseudocode on the structure? Say I have 4 models, each with their own dataloaders and optimizers as a list.
1 Like