CUDA initialization: CUDA driver initialization failed

vasigorc · June 26, 2025, 1:29pm

Hello! I have a 1 year old laptop from System 76 that shipped with POP!_OS 22.04 which is a fork of the same Ubuntu version:

cat /etc/os-release
NAME="Pop!_OS"
VERSION="22.04 LTS"
ID=pop
ID_LIKE="ubuntu debian"
PRETTY_NAME="Pop!_OS 22.04 LTS"
VERSION_ID="22.04"
HOME_URL="https://pop.system76.com"
SUPPORT_URL="https://support.system76.com"
BUG_REPORT_URL="https://github.com/pop-os/pop/issues"
PRIVACY_POLICY_URL="https://system76.com/privacy"
VERSION_CODENAME=jammy
UBUNTU_CODENAME=jammy
LOGO=distributor-logo-pop-os

It does have Nvidia GPU:

 nvidia-smi
Thu Jun 26 09:18:42 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.153.02             Driver Version: 570.153.02     CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4080 ...    Off |   00000000:02:00.0 Off |                  N/A |
| N/A   43C    P0            593W /  175W |      15MiB /  12282MiB |     15%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A            3886      G   /usr/lib/xorg/Xorg                        4MiB |
+-----------------------------------------------------------------------------------------+

The default OS CUDA toolkit was of a far inferior version:

apt-cache show nvidia-cuda-toolkit | grep Version
Version: 11.5.1-1ubuntu1

So I had to remove it and followed the official Nvidia installation guide for CUDA 12.8 version:

 9576* sudo apt remove nvidia-cuda-toolkit
 9577* sudo apt autoremove
 9578* wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb
 9579* sudo dpkg -i cuda-keyring_1.0-1_all.deb
 9580* sudo apt update
 9581* sudo apt install cuda-toolkit-12-8

 9608  sudo apt-get install nvidia-gds-12-8

I’ve also followed the post installation guide to make sure that nvcc and all is on the path:

echo $PATH | grep -i cuda
/usr/local/cuda-12.8/bin:/home/vasilegorcinschi/go/bin:/usr/local/go/bin:/usr/local/cuda-12.8/bin:...
...
echo $LD_LIBRARY_PATH
/usr/local/cuda-12.6/lib64
...
nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Fri_Feb_21_20:23:50_PST_2025
Cuda compilation tools, release 12.8, V12.8.93
Build cuda_12.8.r12.8/compiler.35583870_0

Finally, I’ve re-intssalled torch for cu128 (I had previously also installed it when I had 11.5 cudatoolkit) with the command prompted on PyTorch’s home page:

 9638  pip3 uninstall torch torchvision torchaudio -y
 9639  pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
  pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
  Defaulting to user installation because normal site-packages is not writeable
  Looking in indexes: https://download.pytorch.org/whl/cu128
  Collecting torch
    Using cached https://download.pytorch.org/whl/cu128/torch-2.7.1%2Bcu128-cp310-cp310-manylinux_2_28_x86_64.whl (1039.4 MB)
  Collecting torchvision
    Using cached https://download.pytorch.org/whl/cu128/torchvision-0.22.1%2Bcu128-cp310-cp310-manylinux_2_28_x86_64.whl (8.7 MB)
  Collecting torchaudio
    Using cached https://download.pytorch.org/whl/cu128/torchaudio-2.7.1%2Bcu128-cp310-cp310-manylinux_2_28_x86_64.whl (3.9 MB)
  Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.8.61 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.8.61)
  Requirement already satisfied: typing-extensions>=4.10.0 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (4.12.2)
  Requirement already satisfied: nvidia-cufile-cu12==1.13.0.11 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (1.13.0.11)
  Requirement already satisfied: sympy>=1.13.3 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (1.13.3)
  Requirement already satisfied: nvidia-cufft-cu12==11.3.3.41 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (11.3.3.41)
  Requirement already satisfied: nvidia-cusparse-cu12==12.5.7.53 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.5.7.53)
  Requirement already satisfied: filelock in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (3.16.1)
  Requirement already satisfied: nvidia-nvjitlink-cu12==12.8.61 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.8.61)
  Requirement already satisfied: networkx in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (3.3)
  Requirement already satisfied: triton==3.3.1 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (3.3.1)
  Requirement already satisfied: fsspec in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (2024.9.0)
  Requirement already satisfied: nvidia-curand-cu12==10.3.9.55 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (10.3.9.55)
  Requirement already satisfied: nvidia-nvtx-cu12==12.8.55 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.8.55)
  Requirement already satisfied: jinja2 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (3.1.4)
  Requirement already satisfied: nvidia-cublas-cu12==12.8.3.14 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.8.3.14)
  Requirement already satisfied: nvidia-cusparselt-cu12==0.6.3 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (0.6.3)
  Requirement already satisfied: nvidia-cudnn-cu12==9.7.1.26 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (9.7.1.26)
  Requirement already satisfied: nvidia-cusolver-cu12==11.7.2.55 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (11.7.2.55)
  Requirement already satisfied: nvidia-cuda-cupti-cu12==12.8.57 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.8.57)
  Requirement already satisfied: nvidia-nccl-cu12==2.26.2 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (2.26.2)
  Requirement already satisfied: nvidia-cuda-runtime-cu12==12.8.57 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torch) (12.8.57)
  Requirement already satisfied: setuptools>=40.8.0 in /usr/lib/python3/dist-packages (from triton==3.3.1->torch) (59.6.0)
  Requirement already satisfied: numpy in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torchvision) (1.26.4)
  Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from torchvision) (10.1.0)
  Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from sympy>=1.13.3->torch) (1.3.0)
  Requirement already satisfied: MarkupSafe>=2.0 in /home/vasilegorcinschi/.local/lib/python3.10/site-packages (from jinja2->torch) (3.0.2)
  Installing collected packages: torch, torchvision, torchaudio
  Successfully installed torch-2.7.1+cu128 torchaudio-2.7.1+cu128 torchvision-0.22.1+cu128

But when I try to check for CUDA availability I get a failure (just I had it when using the default toolkit):

python -c 'import torch; print(torch.__version__, torch.version.cuda, torch.cuda.is_available())'
/home/vasilegorcinschi/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:174: UserWarning: CUDA initialization: CUDA driver initialization failed, you might not have a CUDA gpu. (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
  2.7.1+cu128 12.8 False

Could you please kindly help me? Any help will be greatly appreciated!

ptrblck · June 26, 2025, 1:59pm

Your NVIDIA driver is not properly installed and you might need to reinstall it. We have also seen other topics here where users ignored the request to reboot the system after upgrading the driver, so you might also want to make sure your laptop was rebooted already.

vasigorc · June 26, 2025, 2:31pm

Hm I installed the driver as instructed by Ssytem76:

sudo apt-get install system76-driver-nvidia
[sudo] password for vasilegorcinschi: 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
system76-driver-nvidia is already the newest version (24.04.5~1749231807~22.04~6e6b806).
The following package was automatically installed and is no longer required:
  dctrl-tools
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Maybe I need to reach out to them.

vasigorc · June 26, 2025, 5:00pm

To be fair, I’ve downloaded and installed cuda-samples and the tests fail for me, e.g.:

./build/Samples/1_Utilities/deviceQuery/deviceQuery
./build/Samples/1_Utilities/deviceQuery/deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 3
-> initialization error
Result = FAIL

FWIW, some people also advised to assure that sudo systemctl status nvidia-fabricmanager is running and enabled, and I don’t have it installed, but I am not sure if I try to install if it won’t make the things even more complicated.

ptrblck · June 26, 2025, 6:13pm

Thanks for confirming the same errors using plain CUDA samples!

Fabric Manager is used for NVSwitch systems. Since your system has a single GPU it won’t be needed and is unrelated to your errors.

vasigorc · July 5, 2025, 7:29pm

@ptrblck could you please confirm/refute whether cuda-toolkit is required for PyTorch to be able to leverage GPU?

ptrblck · July 7, 2025, 12:46pm

The PyTorch binaries ship with their own CUDA runtime dependencies and a locally installed CUDA toolkit is not required unless you build PyTorch from source or a custom CUDA extension.
Since the CUDA Samples already fail, the issue is unrelated to PyTorch and you should check your driver installation.