# How to specify GPU usage? **URL:** \ **Category:** Uncategorized\ **Created:** [March 8, 2017, 4:04am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945 "2017-03-08T04:04:06Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![zhoubinxyz](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zhoubinxyz/32/18924_2.png) [@zhoubinxyz](https://discuss.pytorch.org/u/zhoubinxyz)\ **Post date:** [March 8, 2017, 4:04am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/1 "2017-03-08T04:04:06Z")
I am training different models on different GPUs. I have 4 GPUs indexed as 0,1,2,3 I try this way: ```python model = torch.nn.DataParallel(model, device_ids=[0,1]).cuda() ``` But actual process use GPU index 2,3 instead. and if I use: ```python model = torch.nn.DataParallel(model, device_ids=[1]).cuda() ``` I will get the error: > RuntimeError: Assertion `THCTensor\_(checkGPU)(state, 4, r\_, t, m1, m2)’ failed. at /data/users/soumith/miniconda2/conda-bld/pytorch-cuda80-0.1.8\_1486039719409/work/torch/lib/THC/generic/THCTensorMathBlas.cu:230 How to specify the GPU usage with index? ---
**Author:** ![Seungyoung\_Park](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/seungyoung_park/32/48889_2.png) [@Seungyoung\_Park](https://discuss.pytorch.org/u/Seungyoung_Park)\ **Post date:** [March 8, 2017, 5:39am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/2 "2017-03-08T05:39:41Z")
I am using Ubuntu 16.04. The GPU indexing are the same as you have. If you want to execute [xxx.py](http://xxx.py) using only GPUs 0,1 in Ubuntu 16.04, use the following command as ``` CUDA_VISIBLE_DEVICES=2,3 python xxx.py ``` with nn.DadaParallel in [xxx.py](http://xxx.py). In addition, I don’t think that dataparallel accepts only one gpu. ---
**Author:** ![zhoubinxyz](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zhoubinxyz/32/18924_2.png) [@zhoubinxyz](https://discuss.pytorch.org/u/zhoubinxyz)\ **Post date:** [March 8, 2017, 6:26am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/3 "2017-03-08T06:26:23Z")
Thanks a lot, it works 🙂 Hope pytorch can integrate with this argument to specify gpu usage. ---
**Author:** ![apaszke](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/apaszke/32/21_2.png) [@apaszke](https://discuss.pytorch.org/u/apaszke)\ **Post date:** [March 8, 2017, 10:31am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/4 "2017-03-08T10:31:40Z")
What’s your PyTorch version? It should accept a single GPU. How is that even possible that it uses last two GPUs if you specify `device_ids=[0,1]`? If you run your script with `CUDA_VISIBLE_DEVICES=2,3` it will **always** execute on the last two GPUs, not on the first ones. I can’t see how that helps in this case. `CUDA_VISIBLE_DEVICES=0,1` would make more sense. ---
**Author:** ![Seungyoung\_Park](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/seungyoung_park/32/48889_2.png) [@Seungyoung\_Park](https://discuss.pytorch.org/u/Seungyoung_Park)\ **Post date:** [March 8, 2017, 12:37pm UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/5 "2017-03-08T12:37:05Z")
I am using pytorch 0.1.9 and Ubuntu 16.04. When I use CUDA\_VISIBLE\_DEVICES=2,3 (0,1), ‘nvidia-smi’ tells me that gpus 0,1 (2,3) are used. I do not know the reason, but the gpu id used in nvidia-smi and the gpu id used in pytorch are reversed. You can check it if you use Ubuntu 16.04. ---
**Author:** ![shicai](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shicai/32/221_2.png) [@shicai](https://discuss.pytorch.org/u/shicai)\ **Post date:** [March 8, 2017, 3:02pm UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/6 "2017-03-08T15:02:46Z")
I think it is more likely a cuda/nvidia problem. I have met this problem before when using Caffe with Tesla K10/K80 GPUs. ---
**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\ **Post date:** [March 8, 2017, 3:08pm UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/7 "2017-03-08T15:08:07Z")
@Seungyoung_Park from my experience, it’s usually `nvidia-smi` that is reversed with everything else. For example, on my machine, the numbering from pytorch agrees with the numbering of the `deviceQuery` nvidia sample (and any cuda program for that matter) while `nvidia-smi` is the only one giving a different numbering. ---
**Author:** ![zhoubinxyz](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zhoubinxyz/32/18924_2.png) [@zhoubinxyz](https://discuss.pytorch.org/u/zhoubinxyz)\ **Post date:** [March 10, 2017, 3:26am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/8 "2017-03-10T03:26:08Z")
Thanks a lot for your answering. My pytorch version is 0.1.8. There may be a numbering problem of GPU device, but it does not affect our usages. My problem is about how to allocate GPU usages, now everything is fine 🙂 ---
**Author:** ![jramapuram](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jramapuram/32/4433_2.png) [@jramapuram](https://discuss.pytorch.org/u/jramapuram)\ **Post date:** [July 6, 2017, 12:18am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/9 "2017-07-06T00:18:15Z")
I’m curious about this as well. Can you currently use fractional GPU usage as in tensorflow? The tf equivalent is something like this: ```python with tf.device(FLAGS.device): gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=FLAGS.device_percentage) sess_cfg = tf.ConfigProto(allow_soft_placement=FLAGS.allow_soft_placement, gpu_options=gpu_options) ``` ---
**Author:** ![Brando\_Miranda](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/brando_miranda/32/14355_2.png) [@Brando\_Miranda](https://discuss.pytorch.org/u/Brando_Miranda)\ **Post date:** [August 5, 2017, 6:33pm UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/10 "2017-08-05T18:33:09Z")
How does one use GPUs if one has a custom NN class (that inherits from `torch.nn.Module`)? For example, I know that using the easy example from ([http://pytorch.org/tutorials/beginner/pytorch\_with\_examples.html](http://pytorch.org/tutorials/beginner/pytorch_with_examples.html)) one can just change the type of the tensors being created: ``` dtype = torch.FloatTensor # dtype = torch.cuda.FloatTensor # Uncomment this to run on GPU ``` however, when using things like `torch.nn.Linear` and also `Variable`, how does one make sure to use GPUs? Also, do I really have to track how GPUs are assigned, I am fine with torch just doing its stuff automagically. In particular I would love to see how: [http://pytorch.org/tutorials/beginner/examples\_nn/two\_layer\_net\_module.html#sphx-glr-beginner-examples-nn-two-layer-net-module-py](http://pytorch.org/tutorials/beginner/examples_nn/two_layer_net_module.html#sphx-glr-beginner-examples-nn-two-layer-net-module-py) is made into a GPU version of it. Related SO question: [https://stackoverflow.com/questions/45553613/how-does-one-make-sure-that-everything-is-running-on-gpu-automatically-in-pytorc](https://stackoverflow.com/questions/45553613/how-does-one-make-sure-that-everything-is-running-on-gpu-automatically-in-pytorc) ---
**Author:** ![saurabhjha21](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@saurabhjha21](https://discuss.pytorch.org/u/saurabhjha21)\ **Post date:** [February 12, 2018, 10:18am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/11 "2018-02-12T10:18:23Z")
Dear All, I have installed Nvidia Cuda 9.0 toolkit with Cudnn to my ubuntu machine. I have installed pytorch when i am trying to check for gpu usage by running the below code - ## Code- ## import torch print(torch.rand(2,3).cuda()) I am getting the below error: * * * RuntimeError Traceback (most recent call last) in () ----\> 1 print(torch.rand(2,3).cuda()) ~/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/\_utils.py in _cuda(self, device, async) 67 else: 68 new\_type = getattr(torch.cuda, self. **class**. **name** ) —\> 69 return new\_type(self.size()).copy_(self, async) 70 71 ~/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/cuda/ **init**.py in \_lazy\_new(cls, \*args, \*\*kwargs) 385 # We need this method only for lazy init, so we can remove it 386 del \_CudaBase. **new** –\> 387 return super(\_CudaBase, cls). **new** (cls, \*args, \*\*kwargs) 388 389 RuntimeError: cuda runtime error (2) : out of memory at /opt/conda/conda-bld/pytorch\_1518244421288/work/torch/lib/THC/generic/THCStorage.cu:58 I think pytorch is not communicating with the Nvidia GPU, please advise. Regards Saurabh Jha ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [February 12, 2018, 12:03pm UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/12 "2018-02-12T12:03:40Z")
This error might occur after you installed CUDA etc. without restarting your machine. Have you rebooted after the driver installation? ---
**Author:** ![saurabhjha21](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/45deac/32.png) [@saurabhjha21](https://discuss.pytorch.org/u/saurabhjha21)\ **Post date:** [February 12, 2018, 3:09pm UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/13 "2018-02-12T15:09:29Z")
Yes you are correct, it was fine after i restart the machine ---
**Author:** ![Brando\_Miranda](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/brando_miranda/32/14355_2.png) [@Brando\_Miranda](https://discuss.pytorch.org/u/Brando_Miranda)\ **Post date:** [May 2, 2018, 2:58pm UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/14 "2018-05-02T14:58:56Z")
for a Unix command soln you can also do: ``` export CUDA_VISIBLE_DEVICES=$i ``` though of course that only works if the scripts are independent and stuff like that…otherwise the other solutions here are probably better… ---
**Author:** ![JannaShen](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/aeb1de/32.png) [@JannaShen](https://discuss.pytorch.org/u/JannaShen)\ **Post date:** [May 16, 2018, 10:24am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/15 "2018-05-16T10:24:28Z")
CUDA\_VISIBLE\_DEVICES=$i python main.py ---
**Author:** ![labixiaoK](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/c89c15/32.png) [@labixiaoK](https://discuss.pytorch.org/u/labixiaoK)\ **Post date:** [April 24, 2019, 2:06pm UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/16 "2019-04-24T14:06:42Z")
hi， do you have the answer？ ---
**Author:** ![DonghunP](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/donghunp/32/16163_2.png) [@DonghunP](https://discuss.pytorch.org/u/DonghunP)\ **Post date:** [October 1, 2019, 12:45am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/17 "2019-10-01T00:45:35Z")
Is there anyone who knows that… When I attach below code in python file(in main.py), ```auto import os os.environment["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID" os.environment["CUDA_VISIBLE_DEVICES"] = "0" ``` it does’t work the same as `CUDA_VISIBLE_DEVICES=0 python main.py` do. The former one doesn’t specify(divide) GPU but, the latter one works well. It seems strange to me. Thanks ahead. ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [October 1, 2019, 7:59am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/18 "2019-10-01T07:59:45Z")
I wouldn’t recommend the first approach, since you would have to make sure these lines of code are imported before any other library, which might take the GPU. If some script imports PyTorch and these lines are executed afterwards, they won’t have any effect anymore. The second approach makes sure to mask the devices before running the Python script. ---
**Author:** ![DonghunP](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/donghunp/32/16163_2.png) [@DonghunP](https://discuss.pytorch.org/u/DonghunP)\ **Post date:** [October 1, 2019, 10:14am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/19 "2019-10-01T10:14:04Z")
Totally understand thanks!! ---
**Author:** ![LucasX](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/lucasx/32/7377_2.png) [@LucasX](https://discuss.pytorch.org/u/LucasX)\ **Post date:** [October 9, 2019, 11:27am UTC](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945/20 "2019-10-09T11:27:45Z")
1. try `CUDA_VISIBLE_DEVICES=0,1,2,3 xxx.py` to specify GPU 2. add `os.environ["CUDA_VISIBLE_DEVICES"] = "0,1,2,3"` in you python code [Next page](https://discuss.pytorch.org/t/how-to-specify-gpu-usage/945.md?page=2)