Unable to install pytorch for cuda 12.8

Hey,
I was trying to install pytorch compatible with cuda 12.8 with rtx 5070 ti but this error shows up
Looking in indexes: https://download.pytorch.org/whl/cu128 ERROR: Could not find a version that satisfies the requirement torch (from versions: none) ERROR: No matching distribution found for torch

I have cuda installed on my system
nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2025 NVIDIA Corporation Built on Wed_Jan_15_19:20:09_PST_2025 Cuda compilation tools, release 12.8, V12.8.61 Build cuda_12.8.r12.8/compiler.35404655_0
In a different conda env I have torch with cu128 installed but not able to install in other environments. How to fix it?

Thanks in advance!

Which Python version are you using in the working and failing environments?

I’m using Python 3.8.20 in failing and Python 3.9.21 in working

Our binaries support Python 3.9-3.13, so you would need to update your Python version.

`
NVIDIA GeForce RTX 5070 Ti with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90.
If you want to use the NVIDIA GeForce RTX 5070 Ti GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
`

I have this issue cuda 12.4, python 3.12, ubuntu 22 and rtx 5070

1 Like

You need to install the latest stable or nightly PyTorch binary with CUDA 12.8.

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128. does not support sm_120

It does. Your screenshot shows an older PyTorch 2.5.1 binary with CUDA 12.1.

Thanks, I tried several ways, this link works pip install --pre torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128, but now i am unable to use llama ccp, it does not see the GPU. Any suggestions ?, i tried compile severel times but i get an 128 bit error i cant fix.

regards !

I would recommend asking in the corresponding repository as I don’t know which code base you are using and what “128 bit error” means.

So far im still stuck here.