How to assign GPU in pytorch when CUDA_VISIBLE_DEVICES is not used?

1

I have 4 GPU. If I do not use CUDA_VISIBLE_DEVICES and run such shell:

import torch
a = torch.tensor([1,1], device=“cuda:1”)
a.device
device(type=‘cuda’, index=1)

Then I check the GUP usage by nvidia-smi. I get such result:
±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 15254 C python 519MiB |
| 1 15254 C python 519MiB |
±----------------------------------------------------------------------------+

It seems that my python shell (PID=15254) uses 2 GPU! I’m wondering why the first GPU is used?

2

CUDA context are created on both GPU 0 and GPU 1. All further operations, if only involving GPU 1 tensors, will not use memory on GPU 0.

The context thing is fixed on GitHub master. For a workaround you can set CUDA_VISIBLE_DEVICES.

3

Yes, I think setting CUDA_VISIBLE_DEVICES is the best way to use GPU.