RuntimeError: CUDA error: no kernel image is available for execution on the device , when I use cuda in Nvidia Geforce 730 GT

ElectrikVocal_95 · April 5, 2021, 9:21am

I installed Cuda Toolkit 11.1, cnDNN the latest version, and pytorch for CUDA 11.1 from pip.

pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio===0.8.1 -f https://download.pytorch.org/whl/torch_stable.html

I use the Anaconda3 last version with Python 3.8.5

And I use a Nvidia Geforce 730 GT 2Gb DDR3 ver. 128bit

(base) C:\Users\MIPC>python
Python 3.8.5 (default, Sep  3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.current_device()
0
>>> torch.cuda.device(0)
<torch.cuda.device object at 0x0000019765D4E310>
>>> torch.cuda.device_count()
1
>>> torch.cuda.get_device_name(0)
'NVIDIA GeForce GT 730'
>>> torch.cuda.is_available()
True
>>> t = torch.tensor(1, dtype = torch.int32, device = "cuda")
>>> t = torch.tensor([1.1, 2.2, 3.3], dtype = torch.double64, device = "cuda")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'torch' has no attribute 'double64'
>>> t = torch.tensor([
...     [1,1,1,1],
...     [2,2,2,2],
...     [3,3,3,3]
... ], dtype=torch.float32, device = "cuda")
>>> exit()

apparently it works, right?

And then check that

(base) C:\Users\MIPC\Desktop\MATI\Vtuber_HP\VtuberProject\Assets\TrackingBackend>nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Sun_Feb_14_22:08:44_Pacific_Standard_Time_2021
Cuda compilation tools, release 11.2, V11.2.152
Build cuda_11.2.r11.2/compiler.29618528_0

And check the architecture, but I don’t understand what is this?

(base) C:\Users\MIPC\Desktop\MATI\Vtuber_HP\VtuberProject\Assets\TrackingBackend>python
Python 3.8.5 (default, Sep  3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.get_arch_list()
['sm_37', 'sm_50', 'sm_60', 'sm_61', 'sm_70', 'sm_75', 'sm_80', 'sm_86', 'compute_37']
>>>

It’s extrange, because I did not find errors in the installation

(base) C:\Users\MIPC>python
Python 3.8.5 (default, Sep  3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

(base) C:\Users\MIPC>pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio===0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
Looking in links: https://download.pytorch.org/whl/torch_stable.html
Collecting torch==1.8.1+cu111
  Using cached https://download.pytorch.org/whl/cu111/torch-1.8.1%2Bcu111-cp38-cp38-win_amd64.whl (3055.7 MB)
Collecting torchvision==0.9.1+cu111
  Using cached https://download.pytorch.org/whl/cu111/torchvision-0.9.1%2Bcu111-cp38-cp38-win_amd64.whl (1.9 MB)
Collecting torchaudio===0.8.1
  Using cached torchaudio-0.8.1-cp38-none-win_amd64.whl (109 kB)
Requirement already satisfied: typing-extensions in c:\users\mipc\anaconda3\lib\site-packages (from torch==1.8.1+cu111) (3.7.4.3)
Requirement already satisfied: numpy in c:\users\mipc\anaconda3\lib\site-packages (from torch==1.8.1+cu111) (1.19.2)
Requirement already satisfied: pillow>=4.1.1 in c:\users\mipc\anaconda3\lib\site-packages (from torchvision==0.9.1+cu111) (8.0.1)
Installing collected packages: torch, torchvision, torchaudio
Successfully installed torch-1.8.1+cu111 torchaudio-0.8.1 torchvision-0.9.1+

But when i run my code…

Traceback (most recent call last):
  File "main.py", line 41, in <module>
    pose_data = pose_estimator.get_pose_data(img.copy())
  File "C:\Users\MIPC\Desktop\MATI\Vtuber_HP\VtuberProject\Assets\TrackingBackend\utils\pose_estimator.py", line 74, in get_pose_data
    heatmaps, pafs, scale, pad = self.infer_fast(img)
  File "C:\Users\MIPC\Desktop\MATI\Vtuber_HP\VtuberProject\Assets\TrackingBackend\utils\pose_estimator.py", line 49, in infer_fast
    stages_output = self.net(tensor_img)
  File "C:\Users\MIPC\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\MIPC\Desktop\MATI\Vtuber_HP\VtuberProject\Assets\TrackingBackend\emotion_models\with_mobilenet.py", line 134, in forward
    backbone_features = self.model(x)
  File "C:\Users\MIPC\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\MIPC\anaconda3\lib\site-packages\torch\nn\modules\container.py", line 119, in forward
    input = module(input)
  File "C:\Users\MIPC\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\MIPC\anaconda3\lib\site-packages\torch\nn\modules\container.py", line 119, in forward
    input = module(input)
  File "C:\Users\MIPC\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\MIPC\anaconda3\lib\site-packages\torch\nn\modules\conv.py", line 399, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "C:\Users\MIPC\anaconda3\lib\site-packages\torch\nn\modules\conv.py", line 395, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: CUDA error: no kernel image is available for execution on the device

what should I do?

ptrblck · April 6, 2021, 7:51am

The GT 730 has a compute capability 3.5, which isn’t shipped in the prebuilt binaries anymore, so you could build PyTorch from source as described here.

ElectrikVocal_95 · April 6, 2021, 11:42am

I’m sorry but I really don’t understand what should I do with what they say on that page?

I use CUDA Toolkit 11.1 and the cuDNN of that version. But should I download them or leave them as I have them?

NVIDIA CUDA 9.2 or above
NVIDIA cuDNN v7 or above

And with the compiler what i need do? What do I need to download? (As it says that “or above” version I guess it would be fine, right? )

Compiler compatible with CUDA Note: You could refer to the cuDNN Support Matrix for cuDNN versions with the various supported CUDA, CUDA driver and NVIDIA hardwares

And regarding this …

Would I need to disable CUDA Support?

If you want to disable CUDA support, export environment variable USE_CUDA=0. Other potentially useful environment variables may be found in setup.py.

From Jetson Nano I guess I don’t need anything…

If you are building for NVIDIA’s Jetson platforms (Jetson Nano, TX1, TX2, AGX Xavier), Instructions to install PyTorch for Jetson Nano are available here

From the ROCm compiler do I need something? or I not need disable nothing?

If you want to compile with ROCm support, install
AMD ROCm 4.0 and above installation
ROCm is currently supported only for Linux system.

If you want to disable ROCm support, export environment variable USE_ROCM=0. Other potentially useful environment variables may be found in setup.py.

If I understood correctly, I should only install this with conda, right?

conda install numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing_extensions future six requests dataclasses

or…

conda install -c conda-forge libuv=1.39

is it possible to do this without git? Just downloading the project and running from my terminal?

git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
# if you are updating an existing checkout
git submodule sync
git submodule update --init --recursive

ptrblck · April 6, 2021, 8:09pm

If you’ve already installed a local CUDA toolkit (with cudnn), you can just use it and don’t need to reinstall it.

The CUDA toolkit installs the CUDA compiler (nvcc) and you could check, if it’s working by either compiling the CUDA Samples or checking if it can be found and used via nvcc --version.

If you want to use the GPU (I assume you want to), you should not disable CUDA support.
Otherwise, you could also install the CPU-only binaries and wouldn’t need to build PyTorch from source.

No, you don’t need to build for Jetson.

No, ROCm is used for AMD GPUs.

Yes, you would need some dependencies to build PyTorch from source. If you miss to install a dependency, the installation will yield an error pointing to the missing package.

You might be able to download the repository from GitHub directly, but I’m not sure if this would also download all third-party modules, so I would recommend to use git.

ElectrikVocal_95 · April 7, 2021, 10:52am

In case I uninstalled pytorch, torchvision and torchaudio before doing the process, I don’t know if it was ok.

(base) C:\Users\MIPC>pip uninstall torch
Found existing installation: torch 1.8.1+cu111
Uninstalling torch-1.8.1+cu111:
  Would remove:
    c:\users\mipc\anaconda3\lib\site-packages\caffe2\*
    c:\users\mipc\anaconda3\lib\site-packages\torch-1.8.1+cu111.dist-info\*
    c:\users\mipc\anaconda3\lib\site-packages\torch\*
    c:\users\mipc\anaconda3\scripts\convert-caffe2-to-onnx.exe
    c:\users\mipc\anaconda3\scripts\convert-onnx-to-caffe2.exe
Proceed (y/n)? y
  Successfully uninstalled torch-1.8.1+cu111

(base) C:\Users\MIPC>pip uninstall torchvision
Found existing installation: torchvision 0.9.1+cu111
Uninstalling torchvision-0.9.1+cu111:
  Would remove:
    c:\users\mipc\anaconda3\lib\site-packages\torchvision-0.9.1+cu111.dist-info\*
    c:\users\mipc\anaconda3\lib\site-packages\torchvision\*
Proceed (y/n)? y
  Successfully uninstalled torchvision-0.9.1+cu111

(base) C:\Users\MIPC>pip uninstall torchaudio
Found existing installation: torchaudio 0.8.1
Uninstalling torchaudio-0.8.1:
  Would remove:
    c:\users\mipc\anaconda3\lib\site-packages\torchaudio-0.8.1.dist-info\*
    c:\users\mipc\anaconda3\lib\site-packages\torchaudio\*
Proceed (y/n)? y
  Successfully uninstalled torchaudio-0.8.1

And then I followed this process using git, in the Anaconda Prompt

git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
git submodule sync
git submodule update --init --recursive

Now, should I reinstall Pytorch from the link on its official website and run programs from Anaconda Prompt? Or do I need to do something else before reinstalling pytorch?

In this case, do you recommend the stable version?

pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio===0.8.1 -f https://download.pytorch.org/whl/torch_stable.html

or the nightly version ?

pip install --pre torch torchvision torchaudio -f https://download.pytorch.org/whl/nightly/cu111/torch_nightly.html

Is it really necessary to use Visual Studio?
I have Visual Studio 2019 ver. 16.8.30804.86

" Note: There’s a compilation issue in several Visual Studio 2019 versions since 16.7.1, so please make sure your Visual Studio 2019 version is not in 16.7.1 ~ 16.7.5" In theory it should work, right?

To run programs using CUDA from the console, should I do this?

NVTX is needed to build Pytorch with CUDA. NVTX is a part of CUDA distributive, where it is called "Nsight Compute". To install it onto already installed CUDA run CUDA installation once again and check the corresponding checkbox. Make sure that CUDA with Nsight Compute is installed after Visual Studio.

Currently, VS 2017 / 2019, and Ninja are supported as the generator of CMake. If ninja.exe is detected in PATH, then Ninja will be used as the default generator, otherwise, it will use VS 2017 / 2019.
If Ninja is selected as the generator, the latest MSVC will get selected as the underlying toolchain.

Where should I download these libraries?

Additional libraries such as Magma, oneDNN, a.k.a MKLDNN or DNNL, and Sccache are often needed. Please refer to the installation-helper to install them.

Or only use this commands in the prompt or cmd, and then pytorch will be activate?

cmd

:: [Optional] If you want to build with the VS 2017 generator for old CUDA and PyTorch, please change the value in the next line to `Visual Studio 15 2017`.
:: Note: This value is useless if Ninja is detected. However, you can force that by using `set USE_NINJA=OFF`.
set CMAKE_GENERATOR=Visual Studio 16 2019

:: Read the content in the previous section carefully before you proceed.
:: [Optional] If you want to override the underlying toolset used by Ninja and Visual Studio with CUDA, please run the following script block.
:: "Visual Studio 2019 Developer Command Prompt" will be run automatically.
:: Make sure you have CMake >= 3.12 before you do this when you use the Visual Studio generator.
set CMAKE_GENERATOR_TOOLSET_VERSION=14.27
set DISTUTILS_USE_SDK=1
for /f "usebackq tokens=*" %i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -version [15^,16^) -products * -latest -property installationPath`) do call "%i\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%CMAKE_GENERATOR_TOOLSET_VERSION%

:: [Optional] If you want to override the CUDA host compiler
set CUDAHOSTCXX=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64\cl.exe

python setup.py install

cpeters · April 7, 2021, 11:34am

The point is that the version you would install normally (e.g. a precompiled binary) lacks the support for such an old GPU, so to specifically make use of the GT730 you need to build the program from scratch with that support baked in; at no point would you be downloading a full version to install.

To answer some of the other points: you would be best served with the stable version, as you don’t want an unstable build with still-in-testing features you won’t be using; Visual Studio is necessary to provide a compilation environment on windows. If you want your programs to run using GPU acceleration (yes), then you need Nsight installed as it suggests.

ElectrikVocal_95 · April 7, 2021, 11:45am

How do I install Nsight Compute (NVTX) on my pc?
On this page appears this code with 2 functions

https://docs.nvidia.com/gameworks/content/gameworkslibrary/nvtx/nvidia_tools_extension_library_nvtx.htm

cpeters · April 7, 2021, 11:49am

Run the normal CUDA installation and it should give you checkboxes to install Nsight Compute.

To install it onto already installed CUDA run CUDA installation once again and check the corresponding checkbox.

ElectrikVocal_95 · April 7, 2021, 12:03pm

I think I have Nsight Compute installed, right?

Apparently I installed it without realizing it, but I think it’s that.

Now with Visual Studio 16.8. and Nsight , should I reinstall pytorch (stable version for CUDA 11.1) with Anaconda prompt and pip?

these commands need to be entered in cmd before i install pytorch? are they really necessary?

cmd

:: [Optional] If you want to build with the VS 2017 generator for old CUDA and PyTorch, please change the value in the next line to `Visual Studio 15 2017`.
:: Note: This value is useless if Ninja is detected. However, you can force that by using `set USE_NINJA=OFF`.
set CMAKE_GENERATOR=Visual Studio 16 2019

:: Read the content in the previous section carefully before you proceed.
:: [Optional] If you want to override the underlying toolset used by Ninja and Visual Studio with CUDA, please run the following script block.
:: "Visual Studio 2019 Developer Command Prompt" will be run automatically.
:: Make sure you have CMake >= 3.12 before you do this when you use the Visual Studio generator.
set CMAKE_GENERATOR_TOOLSET_VERSION=14.27
set DISTUTILS_USE_SDK=1
for /f "usebackq tokens=*" %i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -version [15^,16^) -products * -latest -property installationPath`) do call "%i\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%CMAKE_GENERATOR_TOOLSET_VERSION%

:: [Optional] If you want to override the CUDA host compiler
set CUDAHOSTCXX=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64\cl.exe

python setup.py install

cpeters · April 7, 2021, 12:14pm

You are aware that the instructions you’re following are to compile the program from the source code, with all that entails? No current version of pytorch you can possibly install will have the feature you’re after (old compute capability support), so the entire backend of the program needs to be built again to add it in, then that version you have just built will be installed instead of the online ones.

If you don’t follow the instructions, then it will either fail to build or be missing features, which would defeat the objective.

The only thing that you should be changing in these settings is (I think) wherever the desired compute capability is specified.

One problem you might face is lack of support for the current CUDA version, with the added problem that the 730 is old enough and low enough in Nvidia’s product stack that they no longer have a page for it.

ElectrikVocal_95 · April 7, 2021, 1:09pm

In cmd

(base) C:\Users\MIPC\pytorch>set CMAKE_GENERATOR=Visual Studio 16 2019

(base) C:\Users\MIPC\pytorch>set CMAKE_GENERATOR_TOOLSET_VERSION=14.27

(base) C:\Users\MIPC\pytorch>set DISTUTILS_USE_SDK=1

(base) C:\Users\MIPC\pytorch>for /f "usebackq tokens=*" %i in ("%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -version [15^,16^) -products * -latest -property installationPath) do call "%i\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%CMAKE_GENERATOR_TOOLSET_VERSION%
El sistema no puede encontrar el archivo -version.

(base) C:\Users\MIPC\pytorch>set CUDAHOSTCXX=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64\cl.exe

(base) C:\Users\MIPC\pytorch>python setup.py install
Building wheel torch-1.9.0a0+gitbc05867
-- Building version 1.9.0a0+gitbc05867
cmake -GNinja -DBUILD_PYTHON=True -DBUILD_TEST=True -DCMAKE_BUILD_TYPE=Release -DCMAKE_GENERATOR=Visual Studio 16 2019 -DCMAKE_GENERATOR_TOOLSET_VERSION=14.27 -DCMAKE_INSTALL_PREFIX=C:\Users\MIPC\pytorch\torch -DCMAKE_PREFIX_PATH=C:\Users\MIPC\anaconda3\Lib\site-packages -DNUMPY_INCLUDE_DIR=C:\Users\MIPC\anaconda3\lib\site-packages\numpy\core\include -DPYTHON_EXECUTABLE=C:\Users\MIPC\anaconda3\python.exe -DPYTHON_INCLUDE_DIR=C:\Users\MIPC\anaconda3\include -DPYTHON_LIBRARY=C:\Users\MIPC\anaconda3/libs/python38.lib -DTORCH_BUILD_VERSION=1.9.0a0+gitbc05867 -DUSE_NUMPY=True C:\Users\MIPC\pytorch
CMake Error: Error: generator : Ninja
Does not match the generator used previously: Visual Studio 16 2019
Either remove the CMakeCache.txt file and CMakeFiles directory or choose a different binary directory.
Traceback (most recent call last):
  File "setup.py", line 876, in <module>
    build_deps()
  File "setup.py", line 354, in build_deps
    build_caffe2(version=version,
  File "C:\Users\MIPC\pytorch\tools\build_pytorch_libs.py", line 50, in build_caffe2
    cmake.generate(version,
  File "C:\Users\MIPC\pytorch\tools\setup_helpers\cmake.py", line 329, in generate
    self.run(args, env=my_env)
  File "C:\Users\MIPC\pytorch\tools\setup_helpers\cmake.py", line 140, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "C:\Users\MIPC\anaconda3\lib\subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '-GNinja', '-DBUILD_PYTHON=True', '-DBUILD_TEST=True', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_GENERATOR=Visual Studio 16 2019', '-DCMAKE_GENERATOR_TOOLSET_VERSION=14.27', '-DCMAKE_INSTALL_PREFIX=C:\\Users\\MIPC\\pytorch\\torch', '-DCMAKE_PREFIX_PATH=C:\\Users\\MIPC\\anaconda3\\Lib\\site-packages', '-DNUMPY_INCLUDE_DIR=C:\\Users\\MIPC\\anaconda3\\lib\\site-packages\\numpy\\core\\include', '-DPYTHON_EXECUTABLE=C:\\Users\\MIPC\\anaconda3\\python.exe', '-DPYTHON_INCLUDE_DIR=C:\\Users\\MIPC\\anaconda3\\include', '-DPYTHON_LIBRARY=C:\\Users\\MIPC\\anaconda3/libs/python38.lib', '-DTORCH_BUILD_VERSION=1.9.0a0+gitbc05867', '-DUSE_NUMPY=True', 'C:\\Users\\MIPC\\pytorch']' returned non-zero exit status 1.

Shouldn’t I touch any parameters?

I think something failed here, however I think that in the end the setup.py didn’t work fine, right?

(base) C:\Users\MIPC\pytorch>for /f "usebackq tokens=*" %i in ("%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -version [15^,16^) -products * -latest -property installationPath) do call "%i\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%CMAKE_GENERATOR_TOOLSET_VERSION%
El sistema no puede encontrar el archivo -version.

ElectrikVocal_95 · April 9, 2021, 2:42pm

This error is because it cannot find a version for my video card? Is there a way to fix it? (I already followed all the steps)

(base) C:\Users\MIPC\pytorch>for /f "usebackq tokens=*" %i in ("%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -version [15^,16^) -products * -latest -property installationPath) do call "%i\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%CMAKE_GENERATOR_TOOLSET_VERSION%
The system cannot find the -version file

ptrblck · April 10, 2021, 5:03am

I’m not sure, if this issue is related to the GPU, as the error message seems to point to Visual Studio, so I guess your VS setup might not be working correctly.

Muhammad_Ali · October 26, 2021, 9:29am

Hi @ptrblck , I am having this issue with fastai, and fastaudio. I am using Colab Pro +
These are my versions:

And device is as follows:

ptrblck · October 26, 2021, 9:35am

Your A100 needs CUDA>=11, so you should install the PyTorch binaries with CUDA11.3 in the selection menu.

Muhammad_Ali · October 26, 2021, 11:45am

Thanks alot @ptrblck , but how can I select CUDA11.3 in colab?

Muhammad_Ali · October 26, 2021, 12:08pm

I used following for installation:

!pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html

Now next time, if I get by chance A100, I will check it out.
Thanks again, for helping me out