Pytorch support for sm120

deployment
64

you saved my life. thank you. Works with “GeForce RTX 5050” too!

65

Yes, basically old cuda is not working.

Upgrading PyTorch in the existing venv or system to a CUDA 12.8+ nightly for sm_120 supports does the job.

This are my versions tested.

python -c “import sys;import platform;print(sys.version);print(‘Platform:’,platform.platform())”
3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]

python -c “import torch;print(torch.version, torch.version.cuda, torch.cuda.is_available());print(torch.cuda.get_device_name(0))”
2.9.0.dev20250905+cu128 12.8 True
NVIDIA GeForce RTX 5080

You can reinstall using the wheel specified for the nightly build. But I think the stable against Cuda 12.8 also works fine.

pip uninstall -y torch torchvision

pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu128

66

Hello,

I recently bought an RTX 5060 Ti 16GB (Blackwell architecture) GPU. My system has:

  • NVIDIA Driver: 581.xx (latest, supports CUDA 13)

  • CUDA Toolkit: 13.x

  • Python: 3.10.x

  • Stable Diffusion (Automatic1111 & ComfyUI frontends)

When I run Stable Diffusion with PyTorch, I get the following error:

RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,
so the stacktrace below might be incorrect.

It looks like PyTorch does not yet include kernels for the new SM version of Blackwell GPUs (possibly SM 12.x).

I tried:

  • Installing PyTorch 2.0.1 with CUDA 11.8 → same error

  • Installing PyTorch 2.1.x with CUDA 12.1 → same error

  • Installing latest nightly builds (cu121) → still the same error

  • Running with --skip-torch-cuda-test → skips the check, but still falls back to CPU execution

So, my questions are:

  1. When will official PyTorch builds include RTX 5060 Ti (Blackwell) / CUDA 13 support?

  2. Is there a temporary workaround (nightly wheels, source build with CUDA 13) to enable GPU acceleration?

  3. Do I need to compile PyTorch from source with CUDA 13 myself until official support lands?

Any guidance would be really helpful. Right now Stable Diffusion only runs on CPU, which is very slow compared to GPU.

Thanks!

67

You need to install any of our PyTorch binaries built with CUDA 12.8+ as described in this thread to execute code on your Blackwell GPU.

68

For people running Stable Diffusion and encountering the error:

RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.

while having a working PyTorch install, for example (copied from a mesaage above):

import torch
input_ids = torch.randint(0, 1000, (1, 10), device="cuda")
embedding = torch.nn.Embedding(1000, 64).cuda()
output = embedding(input_ids)
print(output)

The issue is mostly that the working PyTorch version is not installed in the Stable Diffusion WebUI virtual environment. You can verify this by running the snippet above using the Python inside the WebUI venv:

C:\Users...\stable-diffusion-webui\venv\Scripts\python.exe

If it fails there, you need to install the correct PyTorch version in that environment.

On Windows, the steps are:

cd C:\Users...\stable-diffusion-webui\venv\Scripts
.\activate (if PowerShell blocks this, run: Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass)

Then, inside the venv:

pip uninstall torch torchvision torchaudio -y
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu129

It worked for me, hope it helps others.

69

At this point I just build from source because MSVC is MSVC, so my issue on windows is resolved if anyone wants whl or anything. eg I built 2 kernels to port FlashMLA to windows GitHub - IISuperluminaLII/FlashMLA_Windows_sm120: Working source for FlashMLA that works on Windows, more specifically blackwell workstation cards..... I want to blow my brains out, I hate Nvidia I really understand Linus now

71

I walked through few similar threads and stayed with my question:
Can I make it work with python 3.8?
The nightly version can be installed with python 3.10 and above only.
I have RTX5060 and must work with python 3.8 due to company’s legacy code

72

Python-3.8 has reached it’s end of life in October 2024 and thus we stopped building with for this Python version starting with PyTorch 2.5 if I’m not mistaken. This was before CUDA 12.8 with Blackwell support was released and supported on PyTorch 2.7 so you would need to update your Python version.

1 Like
73

Hi, is it possible to use Python-3.9 to work with RTX5090?

74

Yes, PyTorch 2.7.x as well as 2.8.0 both released with CUDA 12.8 and Python 3.9.

Python 3.9 support was dropped in PyTorch 2.9.0.

75

As of today, with an MSI RTX 5080 and Windows 11, it’s still not possible to use ComfyUI and, for example, Z-Image. It always gives a CUDA 120 error. Has anyone managed to make it work? Thanks for your kind responses.

76

You are most likely using an older and unsupported PyTorch binary in your environment as described in e.g. this post.

77

Thank you for the quick response. I tried installing ComfyUI and Z-Image locally, but I’m forced to use the CPU :frowning: because it says that, as of today, my RTX 5080 is not supported on Windows with version 120. I was wondering if anyone has managed to get the GPU working somehow?

78

I have a question, Did no one succeed?

79

Can someone pls help me?
I bought my 5060ti with high hopes, and now i’ve got some troubles - i simply cannot install files for pytorch. All my attemps ended with the same error:
ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
ERROR: No matching distribution found for torch
I simply about to give up

80

Use the install instructions from the Get Started page and make sure you are using a supported Python version.