Torch.cuda.is_available() is False, even if torch.backends.cudnn.enabled is True

Here is my Nvidia-smi output:

±----------------------------------------------------------------------------+
| NVIDIA-SMI 516.01 Driver Version: 516.01 CUDA Version: 11.7 |
|-------------------------------±---------------------±---------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce … WDDM | 00000000:01:00.0 On | N/A |
| 0% 49C P8 33W / 370W | 1713MiB / 10240MiB | 14% Default |
| | | N/A |
±------------------------------±---------------------±---------------------+

And this is my nvcc --version output:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_May__3_19:00:59_Pacific_Daylight_Time_2022
Cuda compilation tools, release 11.7, V11.7.64
Build cuda_11.7.r11.7/compiler.31294372_0

And I have added system variables like below,
CUDA_PATH = D:\CUDA11.7 (where I installed)
CUDA_PATH_V11_7 = D:\CUDA11.7
Path = D:\CUDA11.7\bin, D:\CUDA11.7\libnvvp

And I have download cuDnn and,

  • Copy cudnn64_8.dll to D:\CUDA11.7\bin.
  • Copy cudnn.h to D:\CUDA11.7\include.
  • Copy cudnn.lib to D:\CUDA11.7\lib\x64.

And I installed Pytorch by this command:
conda install pytorch torchvision torchaudio cudatoolkit=11.7 -c pytorch

And final result:
Python executable: D:\Anaconda\envs\LSTM\python.exe
PyTorch version: 2.3.1
Is cuDnn available: True
Is CUDA available: False
CUDA is not available. Please check your CUDA and cuDNN installation.

Really need some help. Stucked for days:(

What is torch.version.cuda returning?

torch.version.cuda returns None

In this case the CPU-only PyTorch binary was installed and you should install the binary with the CUDA runtime dependency.

I used this command line:

conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia

And here is the conda list output:

PS C:\Users\Lvyuche\Documents\GitHub\llm\MLP> conda list

packages in environment at D:\Anaconda\envs\LSTM:

Name Version Build Channel

blas 1.0 mkl
brotli 1.0.9 h2bbff1b_8
brotli-bin 1.0.9 h2bbff1b_8
brotli-python 1.0.9 py38hd77b12b_8
ca-certificates 2024.6.2 h56e8100_0 conda-forge
certifi 2024.6.2 py38haa95532_0
charset-normalizer 3.3.2 pypi_0 pypi
click 8.1.7 pypi_0 pypi
colorama 0.4.6 pypi_0 pypi
contourpy 1.0.5 py38h59b6b97_0
cuda-cccl 12.4.127 0 nvidia
cuda-cudart 11.7.99 0 nvidia
cuda-cudart-dev 11.7.99 0 nvidia
cuda-cupti 11.7.101 0 nvidia
cuda-libraries 11.7.1 0 nvidia
cuda-libraries-dev 11.7.1 0 nvidia
cuda-nvrtc 11.7.99 0 nvidia
cuda-nvrtc-dev 11.7.99 0 nvidia
cuda-nvtx 11.7.91 0 nvidia
cuda-runtime 11.7.1 0 nvidia
cuda-version 11.3 hbc958af_3 conda-forge
cudatoolkit 11.3.1 h59b6b97_2
cudnn 8.9.7.29 he6de189_3 conda-forge
cycler 0.11.0 pyhd3eb1b0_0
filelock 3.14.0 pypi_0 pypi
fonttools 4.51.0 py38h2bbff1b_0
freetype 2.12.1 ha860e81_0
fsspec 2024.6.0 pypi_0 pypi
gensim 4.3.2 pypi_0 pypi
gmpy2 2.1.2 py38h7f96b67_0
huggingface-hub 0.23.3 pypi_0 pypi
icu 73.1 h6c2663c_0
idna 3.7 py38haa95532_0
importlib_resources 6.1.1 py38haa95532_1
intel-openmp 2021.4.0 pypi_0 pypi
jinja2 3.1.4 py38haa95532_0
joblib 1.4.2 pypi_0 pypi
jpeg 9e h2bbff1b_1
kiwisolver 1.4.4 py38hd77b12b_0
krb5 1.20.1 h5b6d351_0
lcms2 2.12 h83e58a3_0
lerc 3.0 hd77b12b_0
libbrotlicommon 1.0.9 h2bbff1b_8
libbrotlidec 1.0.9 h2bbff1b_8
libbrotlienc 1.0.9 h2bbff1b_8
libclang 14.0.6 default_hb5a9fac_1
libclang13 14.0.6 default_h8e68704_1
libcublas 11.10.3.66 0 nvidia
libcublas-dev 11.10.3.66 0 nvidia
libcufft 10.7.2.124 0 nvidia
libcufft-dev 10.7.2.124 0 nvidia
libcurand 10.3.5.147 0 nvidia
libcurand-dev 10.3.5.147 0 nvidia
libcusolver 11.4.0.1 0 nvidia
libcusolver-dev 11.4.0.1 0 nvidia
libcusparse 11.7.4.91 0 nvidia
libcusparse-dev 11.7.4.91 0 nvidia
libdeflate 1.17 h2bbff1b_1
libffi 3.4.4 hd77b12b_1
libjpeg-turbo 2.0.0 h196d8e1_0
libnpp 11.7.4.75 0 nvidia
libnpp-dev 11.7.4.75 0 nvidia
libnvjpeg 11.8.0.2 0 nvidia
libnvjpeg-dev 11.8.0.2 0 nvidia
libpng 1.6.39 h8cc25b3_0
libpq 12.17 h906ac69_0
libtiff 4.5.1 hd77b12b_0
libuv 1.44.2 h2bbff1b_0
libwebp-base 1.3.2 h2bbff1b_0
libzlib 1.2.13 h2466b09_6 conda-forge
libzlib-wapi 1.2.13 h2466b09_6 conda-forge
lz4-c 1.9.4 h2bbff1b_1
markupsafe 2.1.5 pypi_0 pypi
matplotlib 3.7.2 py38haa95532_0
matplotlib-base 3.7.2 py38h4ed8f06_0
mkl 2021.4.0 pypi_0 pypi
mkl-service 2.4.0 py38h2bbff1b_0
mkl_fft 1.3.1 py38h277e83a_0
mkl_random 1.2.2 py38hf11a4ad_0
mpc 1.1.0 h7edee0f_1
mpfr 4.0.2 h62dcd97_1
mpir 3.0.0 hec2e145_1
mpmath 1.3.0 py38haa95532_0
networkx 3.1 py38haa95532_0
nltk 3.8.1 pypi_0 pypi
numpy 1.24.4 pypi_0 pypi
numpy-base 1.24.3 py38h005ec55_0
openjpeg 2.4.0 h4fc8c34_0
openssl 3.3.1 h2466b09_0 conda-forge
packaging 24.0 pypi_0 pypi
pillow 10.3.0 py38h2bbff1b_0
pip 24.0 py38haa95532_0
ply 3.11 py38_0
pyparsing 3.0.9 py38haa95532_0
pyqt 5.15.10 py38hd77b12b_0
pyqt5-sip 12.13.0 py38h2bbff1b_0
pysocks 1.7.1 py38haa95532_0
python 3.8.19 h1aa4202_0
python-dateutil 2.9.0post0 py38haa95532_2
pytorch 2.3.1 py3.8_cpu_0 pytorch
pytorch-cuda 11.7 h16d0643_5 pytorch
pytorch-mutex 1.0 cpu pytorch
pyyaml 6.0.1 py38h2bbff1b_0
qt-main 5.15.2 h19c9488_10
regex 2024.5.15 pypi_0 pypi
requests 2.32.3 pypi_0 pypi
safetensors 0.4.3 pypi_0 pypi
scikit-learn 1.3.2 pypi_0 pypi
scipy 1.10.1 pypi_0 pypi
setuptools 69.5.1 py38haa95532_0
sip 6.7.12 py38hd77b12b_0
six 1.16.0 pyhd3eb1b0_1
smart-open 7.0.4 pypi_0 pypi
sqlite 3.45.3 h2bbff1b_0
sympy 1.12.1 pypi_0 pypi
tbb 2021.12.0 pypi_0 pypi
threadpoolctl 3.5.0 pypi_0 pypi
tokenizers 0.19.1 pypi_0 pypi
tomli 2.0.1 py38haa95532_0
torch 2.3.1 pypi_0 pypi
torchaudio 2.3.1 py38_cpu pytorch
torchvision 0.18.1 py38_cpu pytorch
tornado 6.3.3 py38h2bbff1b_0
tqdm 4.66.4 pypi_0 pypi
transformers 4.41.2 pypi_0 pypi
typing-extensions 4.12.2 pypi_0 pypi
typing_extensions 4.11.0 py38haa95532_0
ucrt 10.0.22621.0 h57928b3_0 conda-forge
unicodedata2 15.1.0 py38h2bbff1b_0
vc 14.2 h2eaa2aa_1
vc14_runtime 14.40.33810 ha82c5b3_20 conda-forge
vs2015_runtime 14.40.33810 h3bf8584_20 conda-forge
wheel 0.43.0 py38haa95532_0
win_inet_pton 1.1.0 py38haa95532_0
wrapt 1.16.0 pypi_0 pypi
xz 5.4.6 h8cc25b3_1
yaml 0.2.5 he774522_0
zipp 3.17.0 py38haa95532_0
zlib 1.2.13 h2466b09_6 conda-forge
zstd 1.5.5 hd43e919_2

Is this the correct version? It still does not work though.

It seems you are mixing different CUDA libs with PyTorch pip wheels as well as CPU-only conda binaries from the main channel as well as conda-forge:

cuda-cccl 12.4.127 0 nvidia
...
cudnn 8.9.7.29 he6de189_3 conda-forge
...
pytorch 2.3.1 py3.8_cpu_0 pytorch # !!! Note the _cpu tag here !!!
...
pytorch-cuda 11.7 h16d0643_5 pytorch
...
pytorch-mutex 1.0 cpu pytorch # !!! Note the _cpu tag here !!!
...
torch 2.3.1 pypi_0 pypi # !!! Note the _pypi tag here !!!
...

I would recommend to abandon this virtual environment and to create a new and empty one where you can try to install a single set of PyTorch binaries.

Sorry to bother you again.
I tried to install Pytorch-GPU version only, but I failed to do so.
I did conda install pytorch-cuda=11.7 which I think only installs the GPU version, but my IDE can not import torch with only this package installed. So I tried another way.
According to the Pytorch site, it uses this command, which I guess also contains the CPU version.

conda install pytorch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 pytorch-cuda=11.7 -c pytorch -c nvidia

I tried so, but it still does not work.
I removed the env and created a new one every time before I installed it.
This is what it looks like after I tried the official command line.

(CUDA11.7) C:\Users\Lvyuche>conda list

packages in environment at D:\Anaconda\envs\CUDA11.7:

Name Version Build Channel

cuda-cccl 12.4.127 0 nvidia
cuda-cudart 11.7.99 0 nvidia
cuda-cudart-dev 11.7.99 0 nvidia
cuda-cupti 11.7.101 0 nvidia
cuda-libraries 11.7.1 0 nvidia
cuda-libraries-dev 11.7.1 0 nvidia
cuda-nvrtc 11.7.99 0 nvidia
cuda-nvrtc-dev 11.7.99 0 nvidia
cuda-nvtx 11.7.91 0 nvidia
cuda-runtime 11.7.1 0 nvidia
libcublas 11.10.3.66 0 nvidia
libcublas-dev 11.10.3.66 0 nvidia
libcufft 10.7.2.124 0 nvidia
libcufft-dev 10.7.2.124 0 nvidia
libcurand 10.3.5.147 0 nvidia
libcurand-dev 10.3.5.147 0 nvidia
libcusolver 11.4.0.1 0 nvidia
libcusolver-dev 11.4.0.1 0 nvidia
libcusparse 11.7.4.91 0 nvidia
libcusparse-dev 11.7.4.91 0 nvidia
libdeflate 1.17 h2bbff1b_1
libffi 3.4.4 hd77b12b_1
libjpeg-turbo 2.0.0 h196d8e1_0
libnpp 11.7.4.75 0 nvidia
libnpp-dev 11.7.4.75 0 nvidia
libnvjpeg 11.8.0.2 0 nvidia
libnvjpeg-dev 11.8.0.2 0 nvidia
pip 24.0 py38haa95532_0
pysocks 1.7.1 py38haa95532_0
python 3.8.19 h1aa4202_0
pytorch 2.3.0 py3.8_cpu_0 pytorch
pytorch-cuda 11.7 h16d0643_5 pytorch
pytorch-mutex 1.0 cpu pytorch
pyyaml 6.0.1 py38h2bbff1b_0
torchaudio 2.3.0 py38_cpu pytorch
torchvision 0.18.0 py38_cpu pytorch
typing_extensions 4.11.0 py38haa95532_0
vs2015_runtime 14.29.30133 h43f2093_3
wheel 0.43.0 py38haa95532_0

I’m not sure where this command comes from as the current release for PyTorch 2.3.0 does not ship with CUDA 11.7 runtime dependencies but with 11.8 (and 12.1):

As expected, the specified binary cannot be found and the CPU-only binary is installed:

pytorch 2.3.0 py3.8_cpu_0 pytorch

So please stick to the supported install commands from the install matrix.

I am having a similar problem, although I am using exactly the command from the selection matrix:

conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

to clean environment. Here’s what got installed:

The following NEW packages will be INSTALLED:

  brotli-python      pkgs/main/linux-64::brotli-python-1.0.9-py39h6a678d5_8
  bzip2              pkgs/main/linux-64::bzip2-1.0.8-h5eee18b_6
  certifi            pkgs/main/linux-64::certifi-2024.6.2-py39h06a4308_0
  charset-normalizer pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0_0
  cuda-cudart        nvidia/linux-64::cuda-cudart-12.1.105-0
  cuda-cupti         nvidia/linux-64::cuda-cupti-12.1.105-0
  cuda-libraries     nvidia/linux-64::cuda-libraries-12.1.0-0
  cuda-nvrtc         nvidia/linux-64::cuda-nvrtc-12.1.105-0
  cuda-nvtx          nvidia/linux-64::cuda-nvtx-12.1.105-0
  cuda-opencl        nvidia/linux-64::cuda-opencl-12.5.39-0
  cuda-runtime       nvidia/linux-64::cuda-runtime-12.1.0-0
  cuda-version       nvidia/noarch::cuda-version-12.5-3
  ffmpeg             pytorch/linux-64::ffmpeg-4.3-hf484d3e_0
  filelock           pkgs/main/linux-64::filelock-3.13.1-py39h06a4308_0
  freetype           pkgs/main/linux-64::freetype-2.12.1-h4a9f257_0
  fsspec             pkgs/main/linux-64::fsspec-2024.3.1-py39h06a4308_0
  gmp                pkgs/main/linux-64::gmp-6.2.1-h295c915_3
  gmpy2              pkgs/main/linux-64::gmpy2-2.1.2-py39heeb90bb_0
  gnutls             pkgs/main/linux-64::gnutls-3.6.15-he1e5248_0
  idna               pkgs/main/linux-64::idna-3.7-py39h06a4308_0
  jinja2             pkgs/main/linux-64::jinja2-3.1.4-py39h06a4308_0
  jpeg               pkgs/main/linux-64::jpeg-9e-h5eee18b_1
  lame               pkgs/main/linux-64::lame-3.100-h7b6447c_0
  lcms2              pkgs/main/linux-64::lcms2-2.12-h3be6417_0
  lerc               pkgs/main/linux-64::lerc-3.0-h295c915_0
  libcublas          nvidia/linux-64::libcublas-12.1.0.26-0
  libcufft           nvidia/linux-64::libcufft-11.0.2.4-0
  libcufile          nvidia/linux-64::libcufile-1.10.0.4-0
  libcurand          nvidia/linux-64::libcurand-10.3.6.39-0
  libcusolver        nvidia/linux-64::libcusolver-11.4.4.55-0
  libcusparse        nvidia/linux-64::libcusparse-12.0.2.55-0
  libdeflate         pkgs/main/linux-64::libdeflate-1.17-h5eee18b_1
  libiconv           pkgs/main/linux-64::libiconv-1.16-h5eee18b_3
  libidn2            pkgs/main/linux-64::libidn2-2.3.4-h5eee18b_0
  libjpeg-turbo      pytorch/linux-64::libjpeg-turbo-2.0.0-h9bf148f_0
  libnpp             nvidia/linux-64::libnpp-12.0.2.50-0
  libnvjitlink       nvidia/linux-64::libnvjitlink-12.1.105-0
  libnvjpeg          nvidia/linux-64::libnvjpeg-12.1.1.14-0
  libpng             pkgs/main/linux-64::libpng-1.6.39-h5eee18b_0
  libprotobuf        pkgs/main/linux-64::libprotobuf-3.20.3-he621ea3_0
  libtasn1           pkgs/main/linux-64::libtasn1-4.19.0-h5eee18b_0
  libtiff            pkgs/main/linux-64::libtiff-4.5.1-h6a678d5_0
  libunistring       pkgs/main/linux-64::libunistring-0.9.10-h27cfd23_0
  libwebp-base       pkgs/main/linux-64::libwebp-base-1.3.2-h5eee18b_0
  lz4-c              pkgs/main/linux-64::lz4-c-1.9.4-h6a678d5_1
  markupsafe         pkgs/main/linux-64::markupsafe-2.1.3-py39h5eee18b_0
  mpc                pkgs/main/linux-64::mpc-1.1.0-h10f8cd9_1
  mpfr               pkgs/main/linux-64::mpfr-4.0.2-hb69a4c5_1
  mpmath             pkgs/main/linux-64::mpmath-1.3.0-py39h06a4308_0
  nettle             pkgs/main/linux-64::nettle-3.7.3-hbbd107a_1
  networkx           pkgs/main/linux-64::networkx-3.2.1-py39h06a4308_0
  openh264           pkgs/main/linux-64::openh264-2.1.1-h4ff587b_0
  openjpeg           pkgs/main/linux-64::openjpeg-2.4.0-h3ad879b_0
  pillow             pkgs/main/linux-64::pillow-10.3.0-py39h5eee18b_0
  pysocks            pkgs/main/linux-64::pysocks-1.7.1-py39h06a4308_0
  pytorch            pkgs/main/linux-64::pytorch-2.3.0-cpu_py39hcb105a3_0
  pytorch-cuda       pytorch/linux-64::pytorch-cuda-12.1-ha16c6d3_5
  pytorch-mutex      pytorch/noarch::pytorch-mutex-1.0-cuda
  requests           pkgs/main/linux-64::requests-2.32.2-py39h06a4308_0
  sympy              pkgs/main/linux-64::sympy-1.12-py39h06a4308_0
  torchaudio         pytorch/linux-64::torchaudio-2.3.0-py39_cu121
  torchvision        pytorch/linux-64::torchvision-0.18.0-py39_cu121
  typing_extensions  pkgs/main/linux-64::typing_extensions-4.11.0-py39h06a4308_0
  urllib3            pkgs/main/linux-64::urllib3-2.2.2-py39h06a4308_0
  zstd               pkgs/main/linux-64::zstd-1.5.5-hc292b87_2

So looking carefully it installs a cpu pytorch and pytorch-cuda, and then

>>> import torch
>>> torch.version.cuda
>>> torch.cuda.is_available()
False
>>> 

Driver is running ok and earlier environments work as expected. I was experiencing some CUDDN related warnings and wanted to update to CUDA12. Have gone through the procedure twice. New Conda env, python 3.10 or 3.9, numpy and then then PyTorch.

Conda list shows

pytorch                   2.3.0           cpu_py39hcb105a3_0  
pytorch-cuda              12.1                 ha16c6d3_5    pytorch
pytorch-mutex             1.0                        cuda    pytorch

Can’t see why I always get a cpu version now with Conda, even when copy-pasting the commands from the install matrix. Even with CUDA 11.8 which I have successfully installed earlier. Clearing conda caches did not help either.

However, I managed to install using pip to a Conda env which hopefully solves my immediate problem.

UPDATE. Now exactly the same Conda command appears to install pytorch with cuda as expected. The only thing I did differently was to install pytorch first thing after creating the env, instead of installing numpy first as I had done earlier. Either that or there was some intermittent problem.

pytorch            pytorch/linux-64::pytorch-2.3.1-py3.10_cuda12.1_cudnn8.9.2_0
  pytorch-cuda       pytorch/linux-64::pytorch-cuda-12.1-ha16c6d3_5