Difference between Cuda:0 vs Cuda with 1 GPU

6

Yes and if you want using multi GPUs you shall check with model.DataParallel() and model.to(device).