Torch being installed with cpu only, even when I have a GPU

[Abhiram0399 (Abhiram)]

The following error is on Windows 11
I am getting the following error: AssertionError: Torch not compiled with CUDA enabled. My GPU drivers are up to date as well.

Steps :

1. I created a new Pytorch environment. For some reason, the command “conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch” is by default installing cpu only versions. I tried removing this using “conda remove cpuonly” but I have this error:

(PyTorchEnv) C:\Users\P.S.Abhiram>conda remove cpuonly
Collecting package metadata (repodata.json): done
Solving environment: \
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:

  - pytorch/win-64::torchaudio==0.10.0=py38_cpu
  - pytorch/win-64::torchvision==0.11.1=py38_cpu
done

PackagesNotFoundError: The following packages are missing from the target environment:
  - cpuonly

2. I also checked if Anaconda is up to date and it is.
3. I have run my code using .cuda() and the error is : AssertionError: Torch not compiled with CUDA enabled.
4. I am also attaching my nvidia smi:

(PyTorchEnv) C:\Users\P.S.Abhiram>nvidia-smi
Mon Oct 25 11:38:08 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 496.13       Driver Version: 496.13       CUDA Version: 11.5     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ... WDDM  | 00000000:01:00.0 Off |                  N/A |
| N/A   44C    P8     6W /  N/A |    181MiB /  4096MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     18712    C+G   ...IA GeForce Experience.exe    N/A      |
+-----------------------------------------------------------------------------+

Following are the steps taken to create a new environment and the packages that are being downloaded:

(base) C:\Users\P.S.Abhiram>conda activate PyTorchEnv2

(PyTorchEnv2) C:\Users\P.S.Abhiram>conda list
# packages in environment at C:\Users\P.S.Abhiram\anaconda3\envs\PyTorchEnv2:
#
# Name                    Version                   Build  Channel

(PyTorchEnv2) C:\Users\P.S.Abhiram>conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - cudatoolkit=11.1

Current channels:

  - https://conda.anaconda.org/pytorch/win-64
  - https://conda.anaconda.org/pytorch/noarch
  - https://repo.anaconda.com/pkgs/main/win-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/win-64
  - https://repo.anaconda.com/pkgs/r/noarch
  - https://repo.anaconda.com/pkgs/msys2/win-64
  - https://repo.anaconda.com/pkgs/msys2/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.



(PyTorchEnv2) C:\Users\P.S.Abhiram>conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: C:\Users\P.S.Abhiram\anaconda3\envs\PyTorchEnv2

  added / updated specs:
    - cudatoolkit=11.3
    - pytorch
    - torchaudio
    - torchvision


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    certifi-2021.10.8          |   py39haa95532_0         152 KB
    mkl-service-2.4.0          |   py39h2bbff1b_0          51 KB
    mkl_fft-1.3.1              |   py39h277e83a_0         139 KB
    mkl_random-1.2.2           |   py39hf11a4ad_0         225 KB
    numpy-1.21.2               |   py39hfca59bb_0          24 KB
    numpy-base-1.21.2          |   py39h0829f74_0         4.4 MB
    pillow-8.4.0               |   py39hd45dc43_0         906 KB
    pip-21.2.4                 |   py39haa95532_0         1.8 MB
    python-3.9.7               |       h6244533_1        16.5 MB
    pytorch-1.10.0             |      py3.9_cpu_0       193.2 MB  pytorch
    setuptools-58.0.4          |   py39haa95532_0         778 KB
    torchaudio-0.10.0          |         py39_cpu         2.1 MB  pytorch
    torchvision-0.11.1         |         py39_cpu         7.2 MB  pytorch
    tzdata-2021a               |       h5d7bf9c_0         111 KB
    wincertstore-0.2           |   py39haa95532_2          15 KB
    ------------------------------------------------------------
                                           Total:       227.5 MB

The following NEW packages will be INSTALLED:

  blas               pkgs/main/win-64::blas-1.0-mkl
  ca-certificates    pkgs/main/win-64::ca-certificates-2021.9.30-haa95532_1
  certifi            pkgs/main/win-64::certifi-2021.10.8-py39haa95532_0
  cpuonly            pytorch/noarch::cpuonly-2.0-0
  cudatoolkit        pkgs/main/win-64::cudatoolkit-11.3.1-h59b6b97_2
  freetype           pkgs/main/win-64::freetype-2.10.4-hd328e21_0
  intel-openmp       pkgs/main/win-64::intel-openmp-2021.3.0-haa95532_3372
  jpeg               pkgs/main/win-64::jpeg-9d-h2bbff1b_0
  libpng             pkgs/main/win-64::libpng-1.6.37-h2a8f88b_0
  libtiff            pkgs/main/win-64::libtiff-4.2.0-hd0e1b90_0
  libuv              pkgs/main/win-64::libuv-1.40.0-he774522_0
  libwebp            pkgs/main/win-64::libwebp-1.2.0-h2bbff1b_0
  lz4-c              pkgs/main/win-64::lz4-c-1.9.3-h2bbff1b_1
  mkl                pkgs/main/win-64::mkl-2021.3.0-haa95532_524
  mkl-service        pkgs/main/win-64::mkl-service-2.4.0-py39h2bbff1b_0
  mkl_fft            pkgs/main/win-64::mkl_fft-1.3.1-py39h277e83a_0
  mkl_random         pkgs/main/win-64::mkl_random-1.2.2-py39hf11a4ad_0
  numpy              pkgs/main/win-64::numpy-1.21.2-py39hfca59bb_0
  numpy-base         pkgs/main/win-64::numpy-base-1.21.2-py39h0829f74_0
  olefile            pkgs/main/noarch::olefile-0.46-pyhd3eb1b0_0
  openssl            pkgs/main/win-64::openssl-1.1.1l-h2bbff1b_0
  pillow             pkgs/main/win-64::pillow-8.4.0-py39hd45dc43_0
  pip                pkgs/main/win-64::pip-21.2.4-py39haa95532_0
  python             pkgs/main/win-64::python-3.9.7-h6244533_1
  pytorch            pytorch/win-64::pytorch-1.10.0-py3.9_cpu_0
  pytorch-mutex      pytorch/noarch::pytorch-mutex-1.0-cpu
  setuptools         pkgs/main/win-64::setuptools-58.0.4-py39haa95532_0
  six                pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_0
  sqlite             pkgs/main/win-64::sqlite-3.36.0-h2bbff1b_0
  tk                 pkgs/main/win-64::tk-8.6.11-h2bbff1b_0
  torchaudio         pytorch/win-64::torchaudio-0.10.0-py39_cpu
  torchvision        pytorch/win-64::torchvision-0.11.1-py39_cpu
  typing_extensions  pkgs/main/noarch::typing_extensions-3.10.0.2-pyh06a4308_0
  tzdata             pkgs/main/noarch::tzdata-2021a-h5d7bf9c_0
  vc                 pkgs/main/win-64::vc-14.2-h21ff451_1
  vs2015_runtime     pkgs/main/win-64::vs2015_runtime-14.27.29016-h5e58377_2
  wheel              pkgs/main/noarch::wheel-0.37.0-pyhd3eb1b0_1
  wincertstore       pkgs/main/win-64::wincertstore-0.2-py39haa95532_2
  xz                 pkgs/main/win-64::xz-5.2.5-h62dcd97_0
  zlib               pkgs/main/win-64::zlib-1.2.11-h62dcd97_4
  zstd               pkgs/main/win-64::zstd-1.4.9-h19a0ad4_0

[ptrblck]

Could you try installing the nightly binaries, as there seems to be an issue finding the right stable torchtext binary with CUDA on Windows?

[JamesDickens (James McCulloch Dickens)]

Is this a Windows 11 specific issue?

[ptrblck]

I don’t know, as I’m not using Windows and cannot find any version specification in the corresponding issue.

[jahnavi]

@ptrblck

,

Hi ptrblc,

I am facing the same pytorch installing CPU version instead of GPU version issue.
Mine is a Windows 10 machine.
following is the thread where I have described the issue.

Pytorch for cuda 10.2

Hi, conda uninstall cpuonly does not work, do I have to use any other command to not install CPU version of PyTorch and install Cuda version I went through these discussions and other resources as well, but I could not seem to get PyTorch to use GPU. Whenever I run this command - python -c 'import torch;print(torch.cuda.is_available())' the output would be false Following are the steps I followed - Installed CUDA toolkit 10.2 , Downloaded CuDNN v8.0.5 and copied its file to the CUDA too…

Hope someone can help in solving this installation issue.

[ptrblck]

CUDA 10.2 is not supported anymore, so pick a CUDA 11 runtime from these install instructions.

[jahnavi]

It is working now thanks! I have only run the following command
pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116

and did not do any other changes as in, I am still using CUDA toolkit 10.2, CuDNN v8.0.5. Would it be alright if I use these older versions of CUDA instead of the latest (version 12) ones?

[ptrblck]

Your local CUDA toolkit (and cuDNN) installation won’t be used unless you are building PyTorch from source or a custom CUDA extension. The PyTorch pip wheels and conda binaries ship with their own CUDA runtime, cuDNN, NCCL etc. (CUDA 11.6 in your case) and will use these. You would only need a properly installed NVIDIA driver to execute PyTorch.