# Torch.cuda.is\_available() kept switching to False **URL:** \ **Category:** Uncategorized\ **Created:** [February 14, 2023, 1:59am UTC](https://discuss.pytorch.org/t/torch-cuda-is-available-kept-switching-to-false/172504 "2023-02-14T01:59:55Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Bowen\_D](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bowen_d/32/57306_2.png) [@Bowen\_D](https://discuss.pytorch.org/u/Bowen_D)\ **Post date:** [February 14, 2023, 1:59am UTC](https://discuss.pytorch.org/t/torch-cuda-is-available-kept-switching-to-false/172504/1 "2023-02-14T01:59:55Z")
Hi team! torch.cuda.is\_available() kept switching to False on me. Can someone please shed some lights on how to debug this, my setup: Nvidia Driver version: 510.108.03 CUDA: 11.6 Pytorch: 1.12.1+cu116 I installed pytorch through pip pip install torch==1.12.1+cu116 --extra-index-url [https://download.pytorch.org/whl/cu116](https://download.pytorch.org/whl/cu116) I also tried ` python -m torch.utils.collect_env`, the output I got when cuda.is\_available() = True ```auto Collecting environment information... PyTorch version: 1.12.1+cu116 Is debug build: False CUDA used to build PyTorch: 11.6 ROCM used to build PyTorch: N/A OS: Ubuntu 20.04.5 LTS (x86_64) GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0 Clang version: Could not collect CMake version: Could not collect Libc version: glibc-2.17 Python version: 3.7.13 (default, Mar 29 2022, 02:18:16) [GCC 7.5.0] (64-bit runtime) Python platform: Linux-5.13.0-1031-aws-x86_64-with-debian-bullseye-sid Is CUDA available: True CUDA runtime version: 11.6.124 GPU models and configuration: Could not collect Nvidia driver version: Could not collect cuDNN version: Probably one of the following: /usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.0 /usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.0 /usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.0 /usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.0 /usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.0 /usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.0 /usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.0 HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True Versions of relevant libraries: [pip3] numpy==1.21.6 [pip3] torch==1.12.1+cu116 [conda] numpy 1.21.6 pypi_0 pypi [conda] torch 1.12.1+cu116 pypi_0 pypi ``` The output I got when cuda.is\_available returned False ```auto Collecting environment information... PyTorch version: 1.12.1+cu116 Is debug build: False CUDA used to build PyTorch: 11.6 ROCM used to build PyTorch: N/A OS: Ubuntu 20.04.5 LTS (x86_64) GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0 Clang version: Could not collect CMake version: Could not collect Libc version: glibc-2.17 Python version: 3.7.13 (default, Mar 29 2022, 02:18:16) [GCC 7.5.0] (64-bit runtime) Python platform: Linux-5.13.0-1031-aws-x86_64-with-debian-bullseye-sid Is CUDA available: False CUDA runtime version: 11.6.124 GPU models and configuration: Could not collect Nvidia driver version: Could not collect cuDNN version: Probably one of the following: /usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.0 /usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.0 /usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.0 /usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.0 /usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.0 /usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.0 /usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.0 HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True Versions of relevant libraries: [pip3] numpy==1.21.6 [pip3] torch==1.12.1+cu116 [conda] numpy 1.21.6 pypi_0 pypi [conda] torch 1.12.1+cu116 pypi_0 pypi ``` Behavior observed: torch.cuda.is\_available() return True for a while and switching to False. I am running this in a Kubernete cluster, I had to bounce the pod to get torch.cuda.is\_available() returns True. I am using AWS g4 instance with NVIDIA T4 GPUs. Thanks in advance! ---
**Author:** ![eqy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/eqy/32/46233_2.png) [@eqy](https://discuss.pytorch.org/u/eqy)\ **Post date:** [February 14, 2023, 4:20am UTC](https://discuss.pytorch.org/t/torch-cuda-is-available-kept-switching-to-false/172504/2 "2023-02-14T04:20:49Z")
Are you observing different behavior on different nodes with the same machine image? In that case I would check if there could be something unusual on the machine e.g., via `nvidia-smi`. ---
**Author:** ![Bowen\_D](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/bowen_d/32/57306_2.png) [@Bowen\_D](https://discuss.pytorch.org/u/Bowen_D)\ **Post date:** [February 14, 2023, 5:45pm UTC](https://discuss.pytorch.org/t/torch-cuda-is-available-kept-switching-to-false/172504/3 "2023-02-14T17:45:33Z")
I am not observing different behavior on different nodes. `torch.cuda.is_available()` switch to False on the same node. I was able to reproduce this on different nodes with same machine image. We don’t have nvidia-smi installed, would that matter for pytorch to detect GPU? I see the following with `nvcc --version` ```auto nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2022 NVIDIA Corporation Built on Tue_Mar__8_18:18:20_PST_2022 Cuda compilation tools, release 11.6, V11.6.124 Build cuda_11.6.r11.6/compiler.31057947_0 ``` ---
**Author:** ![eqy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/eqy/32/46233_2.png) [@eqy](https://discuss.pytorch.org/u/eqy)\ **Post date:** [February 14, 2023, 5:51pm UTC](https://discuss.pytorch.org/t/torch-cuda-is-available-kept-switching-to-false/172504/4 "2023-02-14T17:51:14Z")
It is not a requirement to use PyTorch, but it may be useful to diagnose e.g., driver or hardware issues if those are the root cause.