# Pytorch not recognizing GPU -- CUDA initialization: CUDA driver initialization failed, you might not have a CUDA gpu **URL:** \ **Category:** deployment\ **Created:** [January 18, 2024, 3:19pm UTC](https://discuss.pytorch.org/t/pytorch-not-recognizing-gpu-cuda-initialization-cuda-driver-initialization-failed-you-might-not-have-a-cuda-gpu/195653 "2024-01-18T15:19:59Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![ml-py](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/f0a364/32.png) [@ml-py](https://discuss.pytorch.org/u/ml-py)\ **Post date:** [January 18, 2024, 3:19pm UTC](https://discuss.pytorch.org/t/pytorch-not-recognizing-gpu-cuda-initialization-cuda-driver-initialization-failed-you-might-not-have-a-cuda-gpu/195653/1 "2024-01-18T15:19:59Z")
Hello, I’m having issues getting pytorch to recognize my GPU as whenever I run “torch.cuda.is\_available()”, it says the CUDA driver failed to initialize. Pytorch is installed using pip and I have tried reinstalling different versions of the NVIDIA driver / CUDA toolkit and pytorch version but I always get the same error. Any help would be appreciated. **Here is the output from nvidia-smi:** ![image](https://discuss.pytorch.org/uploads/default/original/3X/f/f/ffb29176058b10ed97de7a4c1d0e2808bd70394e.png) **Here is the output from nvcc -V** nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2022 NVIDIA Corporation Built on Wed\_Sep\_21\_10:33:58\_PDT\_2022 Cuda compilation tools, release 11.8, V11.8.89 Build cuda\_11.8.r11.8/compiler.31833905\_0 **Here is the output from torch envirionment:** PyTorch version: 2.1.2+cu121 Is debug build: False CUDA used to build PyTorch: 12.1 ROCM used to build PyTorch: N/A OS: Ubuntu 22.04.3 LTS (x86\_64) GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Clang version: Could not collect CMake version: Could not collect Libc version: glibc-2.35 Python version: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (64-bit runtime) Python platform: Linux-5.15.0-91-generic-x86\_64-with-glibc2.35 Is CUDA available: False CUDA runtime version: Could not collect CUDA\_MODULE\_LOADING set to: N/A GPU models and configuration: GPU 0: NVIDIA A100 80GB PCIe GPU 1: NVIDIA A100 80GB PCIe GPU 2: NVIDIA A100 80GB PCIe GPU 3: NVIDIA A100 80GB PCIe Nvidia driver version: 530.30.02 cuDNN version: Could not collect HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True CPU: Architecture: x86\_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 43 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 64 On-line CPU(s) list: 0-63 Vendor ID: AuthenticAMD Model name: AMD Ryzen Threadripper PRO 3975WX 32-Cores CPU family: 23 Model: 49 Thread(s) per core: 2 Core(s) per socket: 32 Socket(s): 1 Stepping: 0 Frequency boost: enabled CPU max MHz: 4368.1641 CPU min MHz: 2200.0000 BogoMIPS: 6986.40 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr\_opt pdpe1gb rdtscp lm constant\_tsc rep\_good nopl nonstop\_tsc cpuid extd\_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 sse4\_1 sse4\_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf\_lm cmp\_legacy svm extapic cr8\_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr\_core perfctr\_nb bpext perfctr\_llc mwaitx cpb cat\_l3 cdp\_l3 hw\_pstate ssbd mba ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt\_a rdseed adx smap clflushopt clwb sha\_ni xsaveopt xsavec xgetbv1 cqm\_llc cqm\_occup\_llc cqm\_mbm\_total cqm\_mbm\_local clzero irperf xsaveerptr rdpru wbnoinvd amd\_ppin arat npt lbrv svm\_lock nrip\_save tsc\_scale vmcb\_clean flushbyasid decodeassists pausefilter pfthreshold avic v\_vmsave\_vmload vgif v\_spec\_ctrl umip rdpid overflow\_recov succor smca sme sev sev\_es Virtualization: AMD-V L1d cache: 1 MiB (32 instances) L1i cache: 1 MiB (32 instances) L2 cache: 16 MiB (32 instances) L3 cache: 128 MiB (8 instances) NUMA node(s): 1 NUMA node0 CPU(s): 0-63 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Mitigation; untrained return thunk; SMT enabled with STIBP protection Vulnerability Spec rstack overflow: Mitigation; safe RET Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and \_\_user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, IBPB conditional, STIBP always-on, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected Versions of relevant libraries: [pip3] numpy==1.21.5 [pip3] torch==2.1.2 [pip3] torchaudio==2.1.2 [pip3] torchvision==0.16.2 [pip3] triton==2.1.0 [conda] Could not collect ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [January 18, 2024, 4:09pm UTC](https://discuss.pytorch.org/t/pytorch-not-recognizing-gpu-cuda-initialization-cuda-driver-initialization-failed-you-might-not-have-a-cuda-gpu/195653/2 "2024-01-18T16:09:16Z")
Are you able to run any other CUDA application on this system? Installing the NVIDIA driver and installing the PyTorch binaries should be enough to run PyTorch workloads. A locally installed CUDA toolkit won’t be needed unless you build PyTorch from source or a custom CUDA extension. ---
**Author:** ![ml-py](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/f0a364/32.png) [@ml-py](https://discuss.pytorch.org/u/ml-py)\ **Post date:** [January 18, 2024, 6:24pm UTC](https://discuss.pytorch.org/t/pytorch-not-recognizing-gpu-cuda-initialization-cuda-driver-initialization-failed-you-might-not-have-a-cuda-gpu/195653/3 "2024-01-18T18:24:29Z")
Yes- I can run other CUDA apps on the system without issues. I still get this error if I try to validate the CUDA/GPU config with pytorch ![image](https://discuss.pytorch.org/uploads/default/original/3X/0/e/0ebea2b628324e5e9de105fee3fb4adbbab873df.png) ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [January 18, 2024, 7:03pm UTC](https://discuss.pytorch.org/t/pytorch-not-recognizing-gpu-cuda-initialization-cuda-driver-initialization-failed-you-might-not-have-a-cuda-gpu/195653/4 "2024-01-18T19:03:50Z")
I cannot reproduce any issue using the latest wheels on an A100 server: ```python nvidia-smi Thu Jan 18 19:02:49 2024 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 525.105.17 Driver Version: 525.105.17 CUDA Version: 12.3 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA A100-SXM... Off | 00000000:07:00.0 Off | 0 | | N/A 30C P0 60W / 400W | 0MiB / 81920MiB | 0% Default | | | | Disabled | +-------------------------------+----------------------+----------------------+ | 1 NVIDIA A100-SXM... Off | 00000000:0F:00.0 Off | 0 | | N/A 30C P0 58W / 400W | 0MiB / 81920MiB | 0% Default | | | | Disabled | +-------------------------------+----------------------+----------------------+ ... python -c "import torch; print(torch. __version__ ); print(torch.cuda.is_available()); print(torch.randn(1).cuda())" 2.1.2+cu121 True tensor([1.4417], device='cuda:0') ``` ---
**Author:** ![ml-py](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/f0a364/32.png) [@ml-py](https://discuss.pytorch.org/u/ml-py)\ **Post date:** [January 18, 2024, 7:35pm UTC](https://discuss.pytorch.org/t/pytorch-not-recognizing-gpu-cuda-initialization-cuda-driver-initialization-failed-you-might-not-have-a-cuda-gpu/195653/5 "2024-01-18T19:35:54Z")
This is odd. What driver packages and pytorch did you install? I would like to mimic on mine to see if there any changes. ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [January 18, 2024, 8:31pm UTC](https://discuss.pytorch.org/t/pytorch-not-recognizing-gpu-cuda-initialization-cuda-driver-initialization-failed-you-might-not-have-a-cuda-gpu/195653/6 "2024-01-18T20:31:46Z")
NVIDIA driver `525.105.17` with a simple `pip install torch` installing the latest stable `2.1.2+cu121` binary. ---
**Author:** ![ml-py](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/f0a364/32.png) [@ml-py](https://discuss.pytorch.org/u/ml-py)\ **Post date:** [January 18, 2024, 10:43pm UTC](https://discuss.pytorch.org/t/pytorch-not-recognizing-gpu-cuda-initialization-cuda-driver-initialization-failed-you-might-not-have-a-cuda-gpu/195653/7 "2024-01-18T22:43:01Z")
No luck. Are there any other commands or logs I can review to see why it isn’t detecting? ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [January 18, 2024, 11:26pm UTC](https://discuss.pytorch.org/t/pytorch-not-recognizing-gpu-cuda-initialization-cuda-driver-initialization-failed-you-might-not-have-a-cuda-gpu/195653/8 "2024-01-18T23:26:01Z")
You could check your environment as we’ve seen issues in the past reported here where users were unaware of e.g. `CUDA_VISIBLE_DEVICES` being set to an invalid value, thus blocking the GPU usage. This would of course not explain why standalone CUDA applications work, but still checking env variables as well as `dmesg` might give you a clue. Additionally, you could also pull docker containers with PyTorch installed to check if these would work. ---
**Author:** ![aberoham](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/aberoham/32/75082_2.png) [@aberoham](https://discuss.pytorch.org/u/aberoham)\ **Post date:** [January 13, 2025, 3:26pm UTC](https://discuss.pytorch.org/t/pytorch-not-recognizing-gpu-cuda-initialization-cuda-driver-initialization-failed-you-might-not-have-a-cuda-gpu/195653/9 "2025-01-13T15:26:19Z")
I encountered this same problem over the weekend on a machine with multiple A100’s and the missing piece was to make sure the system was running nvidia-fabricmanager [https://docs.nvidia.com/datacenter/tesla/fabric-manager-user-guide/index.html](https://docs.nvidia.com/datacenter/tesla/fabric-manager-user-guide/index.html) – ```auto # systemctl status nvidia-fabricmanager ● nvidia-fabricmanager.service - NVIDIA fabric manager service Loaded: loaded (/lib/systemd/system/nvidia-fabricmanager.service; disabled; vendor preset: enabled) Active: active (running) since Sat 2025-01-11 17:50:50 UTC; 1 day 21h ago Main PID: 20978 (nv-fabricmanage) Tasks: 19 (limit: 231678) Memory: 13.5M CPU: 1min 31.736s CGroup: /system.slice/nvidia-fabricmanager.service └─20978 /usr/bin/nv-fabricmanager -c /usr/share/nvidia/nvswitch/fabricmanager.cfg Jan 11 17:50:38 dltb-5209 systemd[1]: Starting NVIDIA fabric manager service... Jan 11 17:50:39 dltb-5209 nv-fabricmanager[20978]: Connected to 1 node. Jan 11 17:50:50 dltb-5209 nv-fabricmanager[20978]: Successfully configured all the available GPUs and NVSwitches to route NVLink traffic. Jan 11 17:50:50 dltb-5209 systemd[1]: Started NVIDIA fabric manager service. ``` ---
**Author:** ![ravpatil](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/r/6bbea6/32.png) [@ravpatil](https://discuss.pytorch.org/u/ravpatil)\ **Post date:** [August 29, 2025, 3:52pm UTC](https://discuss.pytorch.org/t/pytorch-not-recognizing-gpu-cuda-initialization-cuda-driver-initialization-failed-you-might-not-have-a-cuda-gpu/195653/10 "2025-08-29T15:52:21Z")
I encountered same problem with single L40S Nvidia device on physical system. The nvidia-smi detects one GPU and when tested within python environment, it fails with CUDA driver initialization failed. ```auto python -c "import torch; print(torch.version.cuda); print(torch.cuda.is_available())" 12.4 /opt/app-root/lib64/python3.11/site-packages/torch/cuda/ __init__.py:129: UserWarning: CUDA initialization: CUDA driver initialization failed, you might not have a CUDA gpu. (Triggered internally at /mnt/work-dir/torch-2.6.0/torch-2.6.0/c10/cuda/CUDAFunctions.cpp:109.) return torch._C._cuda_getDeviceCount() > 0 False ``` Verified that kernel modules for nvidia exists ```auto # lsmod | grep nvidia nvidia_drm 114688 0 nvidia_modeset 1449984 1 nvidia_drm nvidia_uvm 5767168 0 nvidia 8224768 7 nvidia_uvm,nvidia_modeset video 73728 1 nvidia_modeset drm_kms_helper 245760 4 ast,nvidia_drm drm 741376 6 drm_kms_helper,ast,drm_shmem_helper,nvidia,nvidia_drm ``` As this is single GPU, so nvidia-fabricmanager service is not required here. The nvidia-smi is able to detect the single GPU Nvidia L40S, but python environment or any container on the system is unable to detect the GPU driver and fails with CUDA initialization failure as above.