NVIDIA RTX 5070 Oobabooga Error

1

I can load a model, but as soon as I ask a question this is the error

RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Name: torch
Version: 2.6.0.dev20241112+cu121
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: packages@pytorch.org
License: BSD-3-Clause

2

Install a PyTorch binary with CUDA 12.8 and it will work.

3

Thank you for your help. Unfortunately I am getting the same error Here is what I installed pip install --pre torch==2.8.0.dev20250424+cu128 torchvision==0.22.0.dev20250425+cu128 torchaudio==2.6.0dev20250425+cu128 --index-url https://download.pytorch.org/whl/nightly/cu128

I also dowloaded torch-2.8.0.dev20250425+cu128-cp312-cp312-win_amd64.whl and installed and it did not work as well

4

In that case another 3rd party library might not support your Blackwell GPU and you would need to isolate which call fails.