What happens if I pair a GTX1080 with a GTX1080TI to train in Pytorch?

hello everyone,
Is it possible to use two different GPUs for training the same network? I mean in a multi-gpu scenario, or should all the GPUs be the same?
I’m planning to get a new GTX1080TI(from Asus, my GTX1080 is from gigabyte so the brand is also different), but don’t know if I should use each separately or I can use both of them for training. any help is greatly appreciated.

You could use different GPUs, but note that the slowest GPU might be the bottleneck for your overall performance. This means your GTX1080TI would have to wait for your GTX1080 to finish its operations before the next training iteration can be started.

1 Like