I have the opportunity to either upgrade the GPU to a titanRTX or buy a 2nd 2080TI and nvlink together and improve the cooling system (it works out at about the same price).
The question is which is better? a single titan RTX does give the benefit of improving the speed of models by about 10-20% over a single 2080Ti, however this is alot slower than 2 2080TI’s. The next benefit is the significant increase in GPU memort (24GB vs 11GB), however can you use the NVlink and pytorch to combine the 2x2080ti into behaving as a single 22GB 2080Ti, as all the multi GPU benchmarks keep the batch sizes the same for 2x2080ti?
I keep hitting the upper limit of the CUDA memory and the only way to get over this is to keep reducing the batch size, but there will come a point where this is no longer viable.
This leads me to the next question, is the increase in speed on the titan just down to the increased memory size?