I have CUDA 11.4 and I’ve installed pytorch 1.11.0.
I thought these versions where compatible but I get this error when I run a python script :
RuntimeError: CUDA error: no kernel image is available for execution on the device.
When I run t
orch.cuda.is_available() I get
Is the problem coming from the CUDA and pytorch versions I’ve installed ?
I have a NVIDIA RTX A4000.
I guess you’ve installed the pip wheels or conda binaries, which will ship with their own CUDA runtime.
Your locally installed CUDA toolkit will be used if you build PyTorch from source or custom CUDA extensions.
Based on the error message you’ve most likely installed the binary with the CUDA 10.2 runtime while your Ampere GPU needs CUDA >= 11.
Thank you for your answer.
The CUDA Runtime Version that I’ve installed is 11.3.
If it can help, here’s what I get when I display the deviceQuery :
CUDA Device Query (Runtime API) version (CUDART static linking)
Detected 1 CUDA Capable device(s)
Device 0: "NVIDIA RTX A4000"
CUDA Driver Version / Runtime Version 11.4 / 11.3
CUDA Capability Major/Minor version number: 8.6
Total amount of global memory: 16095 MBytes (16876699648 bytes)
(48) Multiprocessors, (128) CUDA Cores/MP: 6144 CUDA Cores
GPU Max Clock rate: 1560 MHz (1.56 GHz)
Memory Clock rate: 7001 Mhz
Memory Bus Width: 256-bit
L2 Cache Size: 4194304 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 1536
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (UVA): Yes
Device supports Compute Preemption: Yes
Supports Cooperative Kernel Launch: Yes
Supports MultiDevice Co-op Kernel Launch: Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 1 / 0
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 11.4, CUDA Runtime Version = 11.3, NumDevs = 1, Device0 = NVIDIA RTX A4000
Result = PASS
I really don’t see where the problem is…
As described before, the binaries will use their own Runtime, which is given by:
and I guess it would return
10.2, which is incompatible with your GPU as it’s missing the desired compute capability.
In this case, select the CUDA 11 version for the pip wheel / conda binaries and reinstall it.
torch.version.cuda returns :
So it’s supposed to be compatible right ?
Yes, 11.3 should work and I also just verified that the binaries with the 11.3 runtime work properly on an A4000 using:
x = torch.randn(1, 3, 224, 224, device='cuda')
conv = torch.nn.Conv2d(3, 3, 3).cuda()
out = conv(x)
NVIDIA RTX A4000
_CudaDeviceProperties(name='NVIDIA RTX A4000', major=8, minor=6, total_memory=16117MB, multi_processor_count=48)
torch.Size([1, 3, 222, 222])
I get the exact same results except for
where I get only
1.11.0 as an output.
Is it a problem ?
Is the code snippet working as well or are you seeing the same error message when executing it?
The code is working properly yes… Do you know if pytorch is available with CUDA 11.6 ?
I’ve tried to change CUDA version to 11.6 and I’ve installed pytorch with the command
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
(the latest version), but it doesn’t seem to work either. So I’m wondering if it needs CUDA 11.3.
In that case I don’t fully understand what is causing the issue with the missing GPU architecture.
Are you using a custom extension and didn’t build it for Ampere GPUs?
Yes, we have internal CUDA 11.6 wheels, which can be downloaded e.g. via:
pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116
but this would not fix the issue you are seeing as you are either using the CUDA 10.2 wheels (installed via a wrongly specified requirement) or are building an extension for older architectures.
Sorry, I don’t really know what a custom extension is.
My script is written in python and runs a flask application. It is a python’s framework used to create web applications easily.
What I’m trying to do is to display some images on this web app. Those images are passed through a neural network.
This code work well on a colleague’s computer who have a different GPU and a different version of CUDA.
I’ve installed CUDA 11.6 following these installation instructions :
sudo sh cuda_11.6.0_510.39.01_linux.run
Hello again !
NVIDIA RTX A4000 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
This was in my console too. Is it what you meant with GPU architectures ?
So I just found the solution to my issue. I had installed the wrong pytorch version for my gpu architecture (I think).
I installed pytorch 1.8.1 which is compatible with the architecture sm_86 and now it’s working fine !
Thank you @ptrblck for helping me !!
I don’t think https://download.pytorch.org/whl/cu116/ available. If you will do get request it access denied while https://download.pytorch.org/whl/cu115 show a listing of all packages.
Ok everyone after reading and rereading this thread trying to figure out what in the heck I did wrong… This is the real answer!
Pip is installing torch from the cache of downloads that you have previously installed.
UNINSTALL torch in Terminal: pip3 uninstall torch
DELETE the pip folder in ~/.cache
Rerun installation of torch and cuda from this link Start Locally | PyTorch
You’re welcome in advance!!
Whatever version I Install in my environment it shows successfully installed but when I check with print(torch.version) it shows me 1.12.1+cu113 every time. can anyone please help me?
You might have multiple PyTorch version installed and
1.12.1+cu113 might be the “default” one your current environment finds. Uninstall all previous installations via
pip uninstall torch -y and
conda uninstall pytorch and rerun these commands until no PyTorch install can be found, then reinstall the desired one.