Pyrtorch CUDA Error with NVIDIA Blackwell RTX Pro 6000

1

Hi, I am stuck at an impasse and unsure how to proceed. The TL;DR is I have a server with 6x NVIDIA Blackwell RTX Pro 6000 running Ubuntu 24.04.1. All of the drivers and toolkits appear to be correctly installed (drivers are 570, CUDA 12.8) as commands like nvidia-smi and nvcc –version work as expected. However, when I run torch.cuda.is_available() it doesn’t recognize any cuda GPUs. Any next steps or suggestions on where to look would be greatly appreciated, thank you!

nvidia-smi output:


Wed Oct  8 20:05:48 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.172.08             Driver Version: 570.172.08     CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX PRO 6000 Blac...    On  |   00000000:06:00.0 Off |                    0 |
| N/A   22C    P8             27W /  600W |       0MiB /  97887MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA RTX PRO 6000 Blac...    On  |   00000000:07:00.0 Off |                    0 |
| N/A   21C    P8             27W /  600W |       0MiB /  97887MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   2  NVIDIA RTX PRO 6000 Blac...    On  |   00000000:84:00.0 Off |                    0 |
| N/A   22C    P8             28W /  600W |       0MiB /  97887MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   3  NVIDIA RTX PRO 6000 Blac...    On  |   00000000:85:00.0 Off |                    0 |
| N/A   22C    P8             28W /  600W |       0MiB /  97887MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   4  NVIDIA RTX PRO 6000 Blac...    On  |   00000000:F4:00.0 Off |                    0 |
| N/A   22C    P8             29W /  600W |       0MiB /  97887MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   5  NVIDIA RTX PRO 6000 Blac...    On  |   00000000:F5:00.0 Off |                    0 |
| N/A   21C    P8             29W /  600W |       0MiB /  97887MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

nvcc –version output:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Fri_Feb_21_20:23:50_PST_2025
Cuda compilation tools, release 12.8, V12.8.93
Build cuda_12.8.r12.8/compiler.35583870_0

Ubuntu version:

No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 24.04.1 LTS
Release:	24.04
Codename:	noble

Python version:

python3 -c "import torch; print(torch.__version__)"
2.7.0

Python Pytorch Output:

Python 3.12.3 (main, Aug 14 2025, 17:47:21) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> print(torch.cuda.is_available())
/usr/lib/python3/dist-packages/torch/cuda/__init__.py:174: UserWarning: CUDA initialization: CUDA driver initialization failed, you might not have a CUDA gpu. (Triggered internally at ./c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
False
>>>
2

torch.__version__ should print the CUDA toolkit version used to built the binary, so e.g. 2.7.0+cu128. However, in your case this is missing and it’s unclear why no +cpu tag is shown either:

python -c "import torch; print(torch.__version__)"
2.7.0+cpu

Based on this it seems you might have installed an unofficial PyTorch binary and should instead install any of our binaries built with CUDA 12.8+ from our install matrix.

3

Thanks for the quick reply, really appreciate your help!

I created a new python venv and installed pytorch from the build matrix. I still got the same result of is_cuda_available() being False (but I do have the +cu now):

python -c "import torch; print(torch.__version__)"
2.8.0+cu128

python -c "import torch; print(torch.cuda.is_available())"
/home/cenith/test/lib/python3.12/site-packages/torch/cuda/__init__.py:182: UserWarning: CUDA initialization: CUDA driver initialization failed, you might not have a CUDA gpu. (Triggered internally at /pytorch/c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
False

Any ideas on what to debug from here? It seems to me like theres some fundamental mismatch happening where the drivers are there and installed properly but Pytorch isn’t able to communicate to them or doesn’t know where to find them or something along those lines.

4

Thanks for the update. It seems your setup isn’t able to communicate with the GPU and you would most likely be able to reproduce the same issue using any other CUDA applications, such as the CUDA samples. If you can reproduce it, check for any Xid errors in dmesg or reinstall the driver.