CUDA 12.6 and torch.cuda.is_available return false

torch.compile
1

Hi,

I have NVIDIA-SMI 560.35.03, Driver Version: 560.35.03, CUDA 12.6 installed in the server. Python version is 3.9.18. But I tried installing torch version 2.5 + cu124; 2.4 + cu121. It’s all return torch.cuda.is_available() is false. ps: torch.backends.cudnn.enabled is true

Really need help please

2

Your NVIDIA driver might not be properly installed if PyTorch cannot communicate with the GPU assuming you’ve installed a CDUA-enabled PyTorch binary (you can check it e.g. via torch.version.cuda).

3

torch.version.cuda
‘12.4’
I think this is cuda 12.4 installed?

4

I checked print(“Torch version:”, torch.version); print(“CUDA version:”, torch.version.cuda) and print(“CUDA available:”, torch.cuda.is_available()). The third one return False.

5

In this case your system seems to have trouble communicating with the device which could happen e.g. if you’ve updated the driver without a restart. Recently, another user explained they were seeing this issue after letting their device sleep and needed to reset the GPU or restart their system.

6

It is always the same …
Everybody needed to fix this problem at least once.
My homemade recipe is:

  1. Delete your conda environment completely
  2. Recreate your conda environment (possibly from file)
  3. Install PyTorch using the conda installer (in your case nightly build)
  4. If torch.cuda.is_available() == False; then:
  5. Repeat step 1, 2 and install PyTorch using pip
7

Hi, thanks for the tips. I have been repeating it 10 times. Still not working… Does it work if I downgrade the system CUDA to 12.4 rather than 12.6?

8

As far as I understood pytorch installs its needed cuda version indipentenly. So, in short, there is no need to downgrade.
I messed up my system so many times, that I would not try to downgrade the driver.
But try at least the following: Install only pytorch with the recipe above without torchaudio and torchvision and see what happens.
So:
$pip3 install torch
or
$conda install pytorch pytorch-cuda=12.4 -c pytorch -c nvidia