# CUDA\_VISIBLE\_DEVICE is of no use **URL:** \ **Category:** Uncategorized\ **Created:** [November 16, 2017, 4:37am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018 "2017-11-16T04:37:15Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![Liang](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/liang/32/2363_2.png) [@Liang](https://discuss.pytorch.org/u/Liang)\ **Post date:** [November 16, 2017, 4:37am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/1 "2017-11-16T04:37:15Z")
I have a 4-titan XP GPU server. When i use os.environ[“CUDA\_VISIBLE\_DEVICES”] =“0,1” to allocate GPUs for a task in python, I find that only GPU 0 is used. And there is out of memroy problems even GPU 1 is free. Should I allocate memory to different GPUs myself? ---
**Author:** ![colesbury](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/colesbury/32/26_2.png) [@colesbury](https://discuss.pytorch.org/u/colesbury)\ **Post date:** [November 16, 2017, 5:31am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/2 "2017-11-16T05:31:05Z")
Use `CUDA_VISIBLE_DEVICES` (not “DEVICE”). You have to set it before you launch the program – you can’t do it from within the program. ---
**Author:** ![Liang](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/liang/32/2363_2.png) [@Liang](https://discuss.pytorch.org/u/Liang)\ **Post date:** [November 16, 2017, 7:13am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/3 "2017-11-16T07:13:17Z")
My bad, there is a typo in my post. But in my code, when i use os.environ[“CUDA\_VISIBLE\_DEVICES”] =“1,2” , only GPU 1 is used. At least, such a line in Python has its own effect. It can control the use of GPUs. However, It is supposed to make GPU 1 and 2 available for the task, but the result is that only GPU 1 is available. Even when GPU 1 is out of memory, GPU 2 is not used. Is there any other switches controlling parallel computing between two GPUs? BTW, another question: Does Pytorch tend to use GPUs one by one or allocate equal memories to GPUs ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [November 16, 2017, 8:18am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/4 "2017-11-16T08:18:05Z")
You can push your data to a specific GPU using `.cuda(gpu_id)`. E.g. you can load a generator network on one GPU and the discriminator to the other. Another option is to use the [DataParallel](http://pytorch.org/docs/master/nn.html#dataparallel-layers-multi-gpu-distributed) module. ---
**Author:** ![Liang](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/liang/32/2363_2.png) [@Liang](https://discuss.pytorch.org/u/Liang)\ **Post date:** [November 16, 2017, 8:59am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/5 "2017-11-16T08:59:55Z")
Do you mean that if I want to use two GPUs at the same time, i have to change my source codes and add parallel-related codes? Or it would just use one GPU at most if not specified. ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [November 16, 2017, 11:30am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/6 "2017-11-16T11:30:07Z")
Basically it is just one line to use DataParallel: ``` net = torch.nn.DataParallel(model, device_ids=[0, 1, 2]) output = net(input_var) ``` Just wrap your model with `DataParallel` and call the returned net on your data. The `device_ids` parameter specifies the used GPUs. ---
**Author:** ![Liang](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/liang/32/2363_2.png) [@Liang](https://discuss.pytorch.org/u/Liang)\ **Post date:** [November 18, 2017, 3:14am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/7 "2017-11-18T03:14:13Z")
It doesn’t work for me. Still, only the first GPU is used. Is there any more tricks? ---
**Author:** ![Liang](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/liang/32/2363_2.png) [@Liang](https://discuss.pytorch.org/u/Liang)\ **Post date:** [November 18, 2017, 7:20am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/8 "2017-11-18T07:20:37Z")
Does data parallel only support more than batch=1? Actually, I only use batch=1. ---
**Author:** ![SimonW](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/simonw/32/1702_2.png) [@SimonW](https://discuss.pytorch.org/u/SimonW)\ **Post date:** [November 18, 2017, 7:21am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/9 "2017-11-18T07:21:25Z")
You need to use \>=#gpu batch set to apply data parallel. As its name suggests, data parallel just pushes computation for different data in a batch to different gpus. ---
**Author:** ![MrTuo](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mrtuo/32/4088_2.png) [@MrTuo](https://discuss.pytorch.org/u/MrTuo)\ **Post date:** [December 11, 2018, 7:17am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/10 "2018-12-11T07:17:03Z")
Hi! I’m adding `os.environ['CUDA_VISIBLE_DEVICES'] = "2"` in my code does not work, the code always select first GPU, however `CUDA_VISIBLE_DEVICES=2 python train.py` works. I find that `os.environ['CUDA_VISIBLE_DEVICES']` in this [code](https://github.com/victoresque/pytorch-template/blob/master/train.py#L68) is work. Do you know why? ---
**Author:** ![pjavia](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pjavia/32/1143_2.png) [@pjavia](https://discuss.pytorch.org/u/pjavia)\ **Post date:** [December 13, 2018, 11:19pm UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/11 "2018-12-13T23:19:33Z")
@MrTuo This is how pytorch 0.4.1 convention works. If you say CUDA\_VISIBLE\_DEVICES=2, 3. Then for pytorch GPU - 2 is cuda:0 and GPU - 3 is cuda:1. Just check your code is consistent with this convention or not? ---
**Author:** ![mbanani](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mbanani/32/1260_2.png) [@mbanani](https://discuss.pytorch.org/u/mbanani)\ **Post date:** [June 26, 2019, 3:25pm UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/12 "2019-06-26T15:25:40Z")
I had this same issue where setting `CUDA_VISIBLE_DEVICES=2 python train.py` works but setting `os.environ['CUDA_VISIBLE_DEVICES'] = "2"` didn’t. The cause of the issue for me was importing the torch packages before setting `os.environ['CUDA_VISIBLE_DEVICES']`, moving it to the top of the file before importing torch solved it. Hope this helps. ---
**Author:** ![RIVERS](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rivers/32/19486_2.png) [@RIVERS](https://discuss.pytorch.org/u/RIVERS)\ **Post date:** [January 9, 2020, 8:24am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/13 "2020-01-09T08:24:26Z")
That’s helpful for me, thanks 3000 times ---
**Author:** ![cheng\_xie](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/cheng_xie/32/20831_2.png) [@cheng\_xie](https://discuss.pytorch.org/u/cheng_xie)\ **Post date:** [February 22, 2020, 2:13am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/14 "2020-02-22T02:13:46Z")
thank you, it works. ---
**Author:** ![MattBC](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/mattbc/32/29831_2.png) [@MattBC](https://discuss.pytorch.org/u/MattBC)\ **Post date:** [May 4, 2021, 1:27am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/15 "2021-05-04T01:27:08Z")
Hey, I have the opposite problem: code is using both of my GPUs by default, no matter what I do. These are different GPU models and I DO NOT want to use them for parallel processing. I’ve tried setting GPU #0 with cuda\_visible\_devices, tried setting it with torch, moved it to beginning of code, nothing is working. Just for the record, I am doing deep learning object detection importing arcgis and torch. Everything else seems to work fine now, until I try to test learning rate and it tells me my GPUs are imbalanced and that I should exclude GPU #1. I never wanted GPU #1 to be utilized in the first place. EDIT: Nevermind, it appears to be working now after I did move it towards beginning of code. I guess I reset the kernel somehow, which made it work. I’m just a rookie 😛 ---
**Author:** ![Lame](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/c57346/32.png) [@Lame](https://discuss.pytorch.org/u/Lame)\ **Post date:** [October 17, 2021, 9:46am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/16 "2021-10-17T09:46:01Z")
When I tried this solution (I have two gpu), it shows an error `RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!` ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [October 18, 2021, 12:30am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/17 "2021-10-18T00:30:28Z")
I’m not sure which solution you are referring to, but the error could be raised, if you manually specify a device inside the model. Could you post an executable code snippet, which would reproduce the issue, so that we could debug it, please? ---
**Author:** ![saurbhc](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/saurbhc/32/52352_2.png) [@saurbhc](https://discuss.pytorch.org/u/saurbhc)\ **Post date:** [August 13, 2022, 8:24pm UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/18 "2022-08-13T20:24:33Z")
Saved 30 hours of my life. Thanks a ton. ---
**Author:** ![Arthur\_Conmy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/arthur_conmy/32/26759_2.png) [@Arthur\_Conmy](https://discuss.pytorch.org/u/Arthur_Conmy)\ **Post date:** [October 15, 2022, 4:17am UTC](https://discuss.pytorch.org/t/cuda-visible-device-is-of-no-use/10018/19 "2022-10-15T04:17:17Z")
It’s also possible to run into this with bad conda environments. For me ```auto import os os.environ["CUDA_VISIBLE_DEVICES"] = "2" # just use one GPU on big machine import torch assert torch.cuda.device_count() == 1 ``` Failed, but it was because my environment was problematic, and only ```auto import torch print(torch.cuda.current_device()) ``` actually raised an error.