PyTorch 2.3 with Cuda 12.4 wont download GPU version

Hi, I cant get pytorch to download the GPU version. I have tried multiple times and am coming up with the same issue.

| NVIDIA-SMI 552.44 Driver Version: 552.44 CUDA Version: 12.4 |
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
| 0 NVIDIA GeForce RTX 2060 WDDM | 00000000:01:00.0 On | N/A |
| N/A 41C P8 7W / 80W | 62MiB / 6144MiB | 0% Default |
| | | N/A |

| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Mar_28_02:30:10_Pacific_Daylight_Time_2024
Cuda compilation tools, release 12.4, V12.4.131
Build cuda_12.4.r12.4/compiler.34097967_0

and in python I run a script and get

AssertionError: Torch not compiled with CUDA enabled
CUDA Available: False
PyTorch version: 2.3.0+cpu
CUDA Version: None
Number of GPUs: 0

I tried installing: pip install torch torchvision torchaudio --extra-index-url

pip install torch torchvision torchaudio --extra-index-url

All with the same outcome

also tried: pip install torch==2.3.0+cu121 torchvision==0.18.0+cu121 torchaudio==2.3.0+cu121 -f

Assuming you are using a supported Python version, just run pip install torch and it’ll install the latest PyTorch release with CUDA dependencies.

Oh, how I wish that were true for aarch64–although admittedly I have not actually tried simply ‘pip install torch’. I’ve looked for the wheels on your site, but I suppose there is always the possibility that I missed them somehow. If that’s the case, please simply point me in the right direction and disregard the rest of this message, heh. A 2.3.0+cu121 aarch64 container appeared briefly on Docker Hub, and I was elated. Then, it disappeared. I know that the NGC containers exist, but they don’t use pytorch releases, have some non-standard names for other packages, and a whole bunch of packages I don’t need. I presume that you guys removed that container for a reason, but if it should indeed be there, please put it back! ;-). The on-demand pricing for GH200s is just too good for me to pass up.


Thanks for the reply, unfortunately I tried this after uninstalling torch and clearing the cache but it still downloads without gpu

Downloading torch-2.3.0-cp310-cp310-win_amd64.whl (159.8 MB)
---------------------------------------- 159.8/159.8 MB 7.9 MB/s eta 0:00:00
WARNING: Ignoring invalid distribution -orch (c:\users\local\programs\python\python310\lib\site-packages)
Installing collected packages: torch
Successfully installed torch-2.3.0

The ARM+CUDA wheels are not published yet, but if you don’t want to use the NGC containers, you can download and install the wheels from the builder jobs, e.g. this one (scroll down and download the wheel for the Python version you are using).

Note that only sm_90 is currently supported in these wheels.