Do you have CUDA_VISIBLE_DEVICES set in the environment from which you launch your program (or by some script that launches your python program)? That makes other devices disappear from subsequent cuda calls, including from pytorch. This illustrates nicely how CUDA_VISIBLE_DEVICES works.