PyTorch runs on CPU instead of 4090, Windows 11

Anton_Uramer · January 31, 2023, 9:55am

I’m using the nightly PyTorch (for CUDA 11.8) installed with conda, conda was installed with the standard visual installer.

python -m torch.utils.collect_env

Collecting environment information...
PyTorch version: 2.0.0.dev20230130
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.25.2
Libc version: N/A

Python version: 3.9.13 (main, Aug 25 2022, 23:51:50) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.22621-SP0
Is CUDA available: False
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090
Nvidia driver version: 528.24
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] numpydoc==1.4.0
[pip3] rotary-embedding-torch==0.2.1
[pip3] torch==2.0.0.dev20230130
[pip3] torchaudio==2.0.0.dev20230130
[pip3] torchvision==0.15.0.dev20230130
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0           haa95532_640
[conda] mkl-service               2.4.0            py39h2bbff1b_0
[conda] mkl_fft                   1.3.1            py39h277e83a_0
[conda] mkl_random                1.2.2            py39hf11a4ad_0
[conda] numpy                     1.20.0                   pypi_0    pypi
[conda] numpy-base                1.21.5           py39hca35cd5_3
[conda] numpydoc                  1.4.0            py39haa95532_0
[conda] pytorch                   2.0.0.dev20230130     py3.9_cpu_0    pytorch-nightly
[conda] pytorch-cuda              11.8                 h8dd9ede_2    pytorch-nightly
[conda] pytorch-mutex             1.0                         cpu    pytorch-nightly
[conda] rotary-embedding-torch    0.2.1                    pypi_0    pypi
[conda] torchaudio                2.0.0.dev20230130        py39_cpu    pytorch-nightly
[conda] torchvision               0.15.0.dev20230130        py39_cpu    pytorch-nightly

Anton_Uramer · January 31, 2023, 10:01am

Using stable PyTorch with CUDA 11.7 has the same issue

Collecting environment information...
PyTorch version: 1.13.1
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.25.2
Libc version: N/A

Python version: 3.9.13 (main, Aug 25 2022, 23:51:50) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.22621-SP0
Is CUDA available: False
CUDA runtime version: 11.7.99
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090
Nvidia driver version: 528.24
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] numpydoc==1.4.0
[pip3] rotary-embedding-torch==0.2.1
[pip3] torch==1.13.1
[pip3] torchaudio==0.13.1
[pip3] torchvision==0.14.1
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0           haa95532_640
[conda] mkl-service               2.4.0            py39h2bbff1b_0
[conda] mkl_fft                   1.3.1            py39h277e83a_0
[conda] mkl_random                1.2.2            py39hf11a4ad_0
[conda] numpy                     1.20.0                   pypi_0    pypi
[conda] numpy-base                1.21.5           py39hca35cd5_3
[conda] numpydoc                  1.4.0            py39haa95532_0
[conda] pytorch                   1.13.1              py3.9_cpu_0    pytorch
[conda] pytorch-cuda              11.7                 h67b0de4_0    pytorch
[conda] pytorch-mutex             1.0                         cpu    pytorch-nightly
[conda] rotary-embedding-torch    0.2.1                    pypi_0    pypi
[conda] torchaudio                0.13.1                 py39_cpu    pytorch
[conda] torchvision               0.14.1                 py39_cpu    pytorch

ptrblck · January 31, 2023, 10:08am

I don’t know why (as I’m not using Windows), but the CPU-only binaries were installed as indicate by the cpu tag:

[conda] pytorch                   2.0.0.dev20230130     py3.9_cpu_0    pytorch-nightly

Maybe try to install the pip wheels and see if this would work.

Anton_Uramer · January 31, 2023, 10:18am

I used

conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch-nightly -c nvidia

and

conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia

to install these versions. Is there a convenient way to prohibit installing the CPU version with Conda?

Anton_Uramer · January 31, 2023, 10:20am

Trying to install the wheels with this command now

pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu118

Will report back how it goes

vince62s · January 31, 2023, 10:24am

there is no wheel for 11.8
You still can install 11.7 wheel on cuda 11.8 but then if you intend to use puytroch 2.0 with sm_89 arch you’ll need to re-install triton on your own for now.

github.com/pytorch/pytorch

Trying pytorch 2.0 snippet from the website - sm89 error

opened 11:41AM - 05 Dec 22 UTC

closed 06:56AM - 11 Jan 23 UTC

vince62s

high priority triaged module: third_party module: inductor

### 🐛 Describe the bug trying the snippet from the website ``` import torch …import torchvision.models as models model = models.resnet18().cuda() optimizer = torch.optim.SGD(model.parameters(), lr=0.01) compiled_model = torch.compile(model) x = torch.randn(16, 3, 224, 224).cuda() optimizer.zero_grad() out = compiled_model(x) out.sum().backward() optimizer.step() ``` I got Value 'sm_89' is not defined for option 'gpu-name' I am indeed using a rtx4090 and cuda 11.6 BUT works fine without torch.compile ### Versions Collecting environment information... PyTorch version: 1.14.0.dev20221205+cu116 Is debug build: False CUDA used to build PyTorch: 11.6 ROCM used to build PyTorch: N/A OS: Ubuntu 20.04.5 LTS (x86_64) GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0 Clang version: Could not collect CMake version: version 3.25.0 Libc version: glibc-2.31 Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] (64-bit runtime) Python platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.29 Is CUDA available: True CUDA runtime version: Could not collect CUDA_MODULE_LOADING set to: LAZY GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090 Nvidia driver version: 525.60.11 cuDNN version: Probably one of the following: /usr/lib/x86_64-linux-gnu/libcudnn.so.8.6.0 /usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.6.0 /usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.6.0 /usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.6.0 /usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.6.0 /usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.6.0 /usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.6.0 HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True Versions of relevant libraries: [pip3] numpy==1.22.3 [pip3] pytorch-lightning==1.7.7 [pip3] torch==1.14.0.dev20221205+cu116 [pip3] torch-tb-profiler==0.4.0 [pip3] torchaudio==0.14.0.dev20221204+cu116 [pip3] torchtext==0.14.0 [pip3] torchtriton==2.0.0+0d7e753227 [pip3] torchvision==0.15.0.dev20221204+cpu [conda] blas 1.0 mkl [conda] cudatoolkit 11.3.1 h2bc3f7f_2 [conda] faiss-gpu 1.7.2 py3.8_h28a55e0_0_cuda11.3 pytorch [conda] libfaiss 1.7.2 hfc2d529_0_cuda11.3 pytorch [conda] mkl 2021.4.0 h06a4308_640 [conda] mkl-service 2.4.0 py38h7f8727e_0 [conda] mkl_fft 1.3.1 py38hd3c417c_0 [conda] mkl_random 1.2.2 py38h51133e4_0 [conda] numpy 1.23.4 py38h14f4228_0 [conda] numpy-base 1.23.4 py38h31eccc5_0 [conda] pytorch 1.11.0 py3.8_cuda11.3_cudnn8.2.0_0 pytorch [conda] pytorch-mutex 1.0 cuda pytorch [conda] torch 1.14.0.dev20221204+cu117 pypi_0 pypi [conda] torchaudio 0.14.0.dev20221204+cu117 pypi_0 pypi [conda] torchtriton 2.0.0+0d7e753227 pypi_0 pypi [conda] torchvision 0.15.0.dev20221204+cpu pypi_0 pypi cc @ezyang @gchanan @zou3519 @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire

Anton_Uramer · January 31, 2023, 10:25am

That works, I’ve confirmed it by running tortoise-tts about 100 times faster than my 7950X did.
Thanks for the help. Maybe Conda shouldn’t be recommended by default on the website?

Collecting environment information...
PyTorch version: 2.0.0.dev20230129+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.25.2
Libc version: N/A

Python version: 3.9.13 (main, Aug 25 2022, 23:51:50) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.22621-SP0
Is CUDA available: True
CUDA runtime version: 11.7.99
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4090
Nvidia driver version: 528.24
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] numpydoc==1.4.0
[pip3] rotary-embedding-torch==0.2.1
[pip3] torch==2.0.0.dev20230129+cu118
[pip3] torchaudio==2.0.0.dev20230130+cu118
[pip3] torchvision==0.15.0.dev20230130+cu118
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0           haa95532_640
[conda] mkl-service               2.4.0            py39h2bbff1b_0
[conda] mkl_fft                   1.3.1            py39h277e83a_0
[conda] mkl_random                1.2.2            py39hf11a4ad_0
[conda] numpy                     1.20.0                   pypi_0    pypi
[conda] numpy-base                1.21.5           py39hca35cd5_3
[conda] numpydoc                  1.4.0            py39haa95532_0
[conda] pytorch-cuda              11.7                 h67b0de4_0    pytorch
[conda] pytorch-mutex             1.0                         cpu    pytorch-nightly
[conda] rotary-embedding-torch    0.2.1                    pypi_0    pypi
[conda] torch                     2.0.0.dev20230129+cu118          pypi_0    pypi
[conda] torchaudio                2.0.0.dev20230130+cu118          pypi_0    pypi
[conda] torchvision               0.15.0.dev20230130+cu118          pypi_0    pypi

Anton_Uramer · January 31, 2023, 10:26am

I see that it still installed CUDA 11.7 though, as vince mentioned