Torch.cuda.is_available() is False, even if torch.backends.cudnn.enabled is True

Louie · June 24, 2024, 7:16pm

Here is my Nvidia-smi output:

And this is my nvcc --version output:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_May__3_19:00:59_Pacific_Daylight_Time_2022
Cuda compilation tools, release 11.7, V11.7.64
Build cuda_11.7.r11.7/compiler.31294372_0

And I have added system variables like below,
CUDA_PATH = D:\CUDA11.7 (where I installed)
CUDA_PATH_V11_7 = D:\CUDA11.7
Path = D:\CUDA11.7\bin, D:\CUDA11.7\libnvvp

And I have download cuDnn and,

Copy cudnn64_8.dll to D:\CUDA11.7\bin.
Copy cudnn.h to D:\CUDA11.7\include.
Copy cudnn.lib to D:\CUDA11.7\lib\x64.

And I installed Pytorch by this command:
conda install pytorch torchvision torchaudio cudatoolkit=11.7 -c pytorch

And final result:
Python executable: D:\Anaconda\envs\LSTM\python.exe
PyTorch version: 2.3.1
Is cuDnn available: True
Is CUDA available: False
CUDA is not available. Please check your CUDA and cuDNN installation.

Really need some help. Stucked for days:(

ptrblck · June 24, 2024, 7:19pm

What is torch.version.cuda returning?

Louie · June 24, 2024, 7:26pm

torch.version.cuda returns None

ptrblck · June 24, 2024, 7:27pm

In this case the CPU-only PyTorch binary was installed and you should install the binary with the CUDA runtime dependency.

Louie · June 24, 2024, 7:34pm

I used this command line:

conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia

And here is the conda list output:

PS C:\Users\Lvyuche\Documents\GitHub\llm\MLP> conda list

packages in environment at D:\Anaconda\envs\LSTM:

Is this the correct version? It still does not work though.

ptrblck · June 24, 2024, 7:56pm

It seems you are mixing different CUDA libs with PyTorch pip wheels as well as CPU-only conda binaries from the main channel as well as conda-forge:

cuda-cccl 12.4.127 0 nvidia
...
cudnn 8.9.7.29 he6de189_3 conda-forge
...
pytorch 2.3.1 py3.8_cpu_0 pytorch # !!! Note the _cpu tag here !!!
...
pytorch-cuda 11.7 h16d0643_5 pytorch
...
pytorch-mutex 1.0 cpu pytorch # !!! Note the _cpu tag here !!!
...
torch 2.3.1 pypi_0 pypi # !!! Note the _pypi tag here !!!
...

I would recommend to abandon this virtual environment and to create a new and empty one where you can try to install a single set of PyTorch binaries.

Louie · June 24, 2024, 9:10pm

Sorry to bother you again.
I tried to install Pytorch-GPU version only, but I failed to do so.
I did conda install pytorch-cuda=11.7 which I think only installs the GPU version, but my IDE can not import torch with only this package installed. So I tried another way.
According to the Pytorch site, it uses this command, which I guess also contains the CPU version.

conda install pytorch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 pytorch-cuda=11.7 -c pytorch -c nvidia

I tried so, but it still does not work.
I removed the env and created a new one every time before I installed it.
This is what it looks like after I tried the official command line.

(CUDA11.7) C:\Users\Lvyuche>conda list

packages in environment at D:\Anaconda\envs\CUDA11.7:

ptrblck · June 25, 2024, 2:26am

I’m not sure where this command comes from as the current release for PyTorch 2.3.0 does not ship with CUDA 11.7 runtime dependencies but with 11.8 (and 12.1):

As expected, the specified binary cannot be found and the CPU-only binary is installed:

pytorch 2.3.0 py3.8_cpu_0 pytorch

So please stick to the supported install commands from the install matrix.

htoyryla · June 28, 2024, 11:26am

I am having a similar problem, although I am using exactly the command from the selection matrix:

conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

to clean environment. Here’s what got installed:

The following NEW packages will be INSTALLED:

  brotli-python      pkgs/main/linux-64::brotli-python-1.0.9-py39h6a678d5_8
  bzip2              pkgs/main/linux-64::bzip2-1.0.8-h5eee18b_6
  certifi            pkgs/main/linux-64::certifi-2024.6.2-py39h06a4308_0
  charset-normalizer pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0_0
  cuda-cudart        nvidia/linux-64::cuda-cudart-12.1.105-0
  cuda-cupti         nvidia/linux-64::cuda-cupti-12.1.105-0
  cuda-libraries     nvidia/linux-64::cuda-libraries-12.1.0-0
  cuda-nvrtc         nvidia/linux-64::cuda-nvrtc-12.1.105-0
  cuda-nvtx          nvidia/linux-64::cuda-nvtx-12.1.105-0
  cuda-opencl        nvidia/linux-64::cuda-opencl-12.5.39-0
  cuda-runtime       nvidia/linux-64::cuda-runtime-12.1.0-0
  cuda-version       nvidia/noarch::cuda-version-12.5-3
  ffmpeg             pytorch/linux-64::ffmpeg-4.3-hf484d3e_0
  filelock           pkgs/main/linux-64::filelock-3.13.1-py39h06a4308_0
  freetype           pkgs/main/linux-64::freetype-2.12.1-h4a9f257_0
  fsspec             pkgs/main/linux-64::fsspec-2024.3.1-py39h06a4308_0
  gmp                pkgs/main/linux-64::gmp-6.2.1-h295c915_3
  gmpy2              pkgs/main/linux-64::gmpy2-2.1.2-py39heeb90bb_0
  gnutls             pkgs/main/linux-64::gnutls-3.6.15-he1e5248_0
  idna               pkgs/main/linux-64::idna-3.7-py39h06a4308_0
  jinja2             pkgs/main/linux-64::jinja2-3.1.4-py39h06a4308_0
  jpeg               pkgs/main/linux-64::jpeg-9e-h5eee18b_1
  lame               pkgs/main/linux-64::lame-3.100-h7b6447c_0
  lcms2              pkgs/main/linux-64::lcms2-2.12-h3be6417_0
  lerc               pkgs/main/linux-64::lerc-3.0-h295c915_0
  libcublas          nvidia/linux-64::libcublas-12.1.0.26-0
  libcufft           nvidia/linux-64::libcufft-11.0.2.4-0
  libcufile          nvidia/linux-64::libcufile-1.10.0.4-0
  libcurand          nvidia/linux-64::libcurand-10.3.6.39-0
  libcusolver        nvidia/linux-64::libcusolver-11.4.4.55-0
  libcusparse        nvidia/linux-64::libcusparse-12.0.2.55-0
  libdeflate         pkgs/main/linux-64::libdeflate-1.17-h5eee18b_1
  libiconv           pkgs/main/linux-64::libiconv-1.16-h5eee18b_3
  libidn2            pkgs/main/linux-64::libidn2-2.3.4-h5eee18b_0
  libjpeg-turbo      pytorch/linux-64::libjpeg-turbo-2.0.0-h9bf148f_0
  libnpp             nvidia/linux-64::libnpp-12.0.2.50-0
  libnvjitlink       nvidia/linux-64::libnvjitlink-12.1.105-0
  libnvjpeg          nvidia/linux-64::libnvjpeg-12.1.1.14-0
  libpng             pkgs/main/linux-64::libpng-1.6.39-h5eee18b_0
  libprotobuf        pkgs/main/linux-64::libprotobuf-3.20.3-he621ea3_0
  libtasn1           pkgs/main/linux-64::libtasn1-4.19.0-h5eee18b_0
  libtiff            pkgs/main/linux-64::libtiff-4.5.1-h6a678d5_0
  libunistring       pkgs/main/linux-64::libunistring-0.9.10-h27cfd23_0
  libwebp-base       pkgs/main/linux-64::libwebp-base-1.3.2-h5eee18b_0
  lz4-c              pkgs/main/linux-64::lz4-c-1.9.4-h6a678d5_1
  markupsafe         pkgs/main/linux-64::markupsafe-2.1.3-py39h5eee18b_0
  mpc                pkgs/main/linux-64::mpc-1.1.0-h10f8cd9_1
  mpfr               pkgs/main/linux-64::mpfr-4.0.2-hb69a4c5_1
  mpmath             pkgs/main/linux-64::mpmath-1.3.0-py39h06a4308_0
  nettle             pkgs/main/linux-64::nettle-3.7.3-hbbd107a_1
  networkx           pkgs/main/linux-64::networkx-3.2.1-py39h06a4308_0
  openh264           pkgs/main/linux-64::openh264-2.1.1-h4ff587b_0
  openjpeg           pkgs/main/linux-64::openjpeg-2.4.0-h3ad879b_0
  pillow             pkgs/main/linux-64::pillow-10.3.0-py39h5eee18b_0
  pysocks            pkgs/main/linux-64::pysocks-1.7.1-py39h06a4308_0
  pytorch            pkgs/main/linux-64::pytorch-2.3.0-cpu_py39hcb105a3_0
  pytorch-cuda       pytorch/linux-64::pytorch-cuda-12.1-ha16c6d3_5
  pytorch-mutex      pytorch/noarch::pytorch-mutex-1.0-cuda
  requests           pkgs/main/linux-64::requests-2.32.2-py39h06a4308_0
  sympy              pkgs/main/linux-64::sympy-1.12-py39h06a4308_0
  torchaudio         pytorch/linux-64::torchaudio-2.3.0-py39_cu121
  torchvision        pytorch/linux-64::torchvision-0.18.0-py39_cu121
  typing_extensions  pkgs/main/linux-64::typing_extensions-4.11.0-py39h06a4308_0
  urllib3            pkgs/main/linux-64::urllib3-2.2.2-py39h06a4308_0
  zstd               pkgs/main/linux-64::zstd-1.5.5-hc292b87_2

So looking carefully it installs a cpu pytorch and pytorch-cuda, and then

>>> import torch
>>> torch.version.cuda
>>> torch.cuda.is_available()
False
>>>

Driver is running ok and earlier environments work as expected. I was experiencing some CUDDN related warnings and wanted to update to CUDA12. Have gone through the procedure twice. New Conda env, python 3.10 or 3.9, numpy and then then PyTorch.

Conda list shows

pytorch                   2.3.0           cpu_py39hcb105a3_0  
pytorch-cuda              12.1                 ha16c6d3_5    pytorch
pytorch-mutex             1.0                        cuda    pytorch

htoyryla · June 28, 2024, 11:44am

Can’t see why I always get a cpu version now with Conda, even when copy-pasting the commands from the install matrix. Even with CUDA 11.8 which I have successfully installed earlier. Clearing conda caches did not help either.

However, I managed to install using pip to a Conda env which hopefully solves my immediate problem.

UPDATE. Now exactly the same Conda command appears to install pytorch with cuda as expected. The only thing I did differently was to install pytorch first thing after creating the env, instead of installing numpy first as I had done earlier. Either that or there was some intermittent problem.

pytorch            pytorch/linux-64::pytorch-2.3.1-py3.10_cuda12.1_cudnn8.9.2_0
  pytorch-cuda       pytorch/linux-64::pytorch-cuda-12.1-ha16c6d3_5

Torch.cuda.is_available() is False, even if torch.backends.cudnn.enabled is True

packages in environment at D:\Anaconda\envs\LSTM:

Name Version Build Channel

packages in environment at D:\Anaconda\envs\CUDA11.7:

Name Version Build Channel