@ptrblck What do you mean by >=5?
Can’t we run this on compute capability of 3.5 as PyTorch officially support 3.5 or higher compute capability right?
Thanks,
Hi @jpeg729 - Pytorch supports compute capability 3.5 or higher right?
I am wondering why it is not working with my GT 730 4GB card. Anything you feel you could suggest to me?
The current PyTorch binaries support compute capability 3.7 and higher. If you are using a GPU with compute capability 3.5, you would have to build form source.
I did that. I followed this document - https://github.com/pytorch/pytorch/#from-source and build it.
I will share you the specs.
I have Nvidia driver 470.x.x with Cuda 11.4.x installed for my UBUNTU 20.04 LTS system.
I didn’t find conda install -c pytorch magma-cuda114 so went with conda install -c pytorch magma-cuda112
Rest all the installation was smooth.
Still, if I try to run something, I get the below-quoted errors:
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
----------------------------
--------------------------------------------------------------------------------------------
home/assertst/.local/lib/python3.8/site-packages/torch/cuda/__init__.py:83: UserWarning:
Found GPU%d %s which is of cuda capability %d.%d.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability supported by this library is %d.%d.
warnings.warn(old_gpu_warn.format(d, name, major, minor, min_arch // 10, min_arch % 10))
Device Used: NVIDIA GeForce GT 730
Capability: (3, 5)
<class 'darknet.Darknet'>
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/assertst/anaconda3/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/home/assertst/anaconda3/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "nephro.py", line 88, in processResult
output = self.model(Variable(img), self.CUDA)
File "/home/assertst/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/assertst/nephroplus/darknet.py", line 319, in forward
x = self.module_list[i](x)
File "/home/assertst/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/assertst/.local/lib/python3.8/site-packages/torch/nn/modules/container.py", line 139, in forward
input = module(input)
File "/home/assertst/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/home/assertst/.local/lib/python3.8/site-packages/torch/nn/modules/activation.py", line 756, in forward
return F.leaky_relu(input, self.negative_slope, self.inplace)
File "/home/assertst/.local/lib/python3.8/site-packages/torch/nn/functional.py", line 1472, in leaky_relu
result = torch._C._nn.leaky_relu_(input, negative_slope)
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
----------------------------
Killing processResult Thread
----------------------------
Do you understand the situation @ptrblck ?
Apparently the build didn’t detect sm_35, so you might need to rebuild it with TORCH_CUDA_ARCH_LIST="3.5" python setup.py install.
I did that too. export TORCH_CUDA_ARCH_LIST="3.5" later I built it with python3 setup.py develop
still getting this error
If the env variable was successfully set and used, print(torch.cuda.get_arch_list()) should return sm_35. Could you check it, please?
I would need to check that on my computer. I am a bit outdoor today.
But I suppose yes, it returns a list. 
This command torch.cuda.get_device_capability(0) outputs (3,5).
I tested it right now. It is printing - [‘sm_37’, ‘sm_50’, ‘sm_60’, ‘sm_70’, ‘sm_75’, ‘sm_80’, ‘sm_86’]
Thanks for the update.
Based on the printed list, it seems you are either using the pre-built CUDA11 binaries, which are using these compute capabilities. This could happen e.g. if you have multiple PyTorch installations in the current environment.
I would recommend to either uninstall torch again, make sure it cannot be found in pip list or conda list, and reinstall it or to create a new virtual environment and reinstall it there.
In either case, check the install log for the used compute capabilities and make sure 3.5 is used.
Okay sure.
This should fix?
Do I need to uninstall the CUDA 11.4 which I have and downgrade? to CUDA 10.x?
Compute capability 3.5 is deprecated in CUDA11, so it should still work to build for this architecture.
Okay. Sure let me try
I uninstalled from pip list. In conda list it is showing torch 1.10.0a0+git2f3300f dev_0 <develop> but I’m unable to remove it.
when trying to build from source, I’m getting this error
-- Found PythonInterp: /home/assertst/anaconda3/bin/python3 (found suitable version "3.8.8", minimum required is "2.7")
-- Could NOT find pybind11 (missing: pybind11_DIR)
-- System pybind11 found
-- pybind11 include dirs: /home/assertst/pytorch/torch/include
-- Adding OpenMP CXX_FLAGS: -fopenmp
-- Will link against OpenMP libraries: /usr/lib/gcc/x86_64-linux-gnu/9/libgomp.so;/usr/lib/x86_64-linux-gnu/libpthread.so
-- Caffe2: CUDA detected: 11.4
-- Caffe2: CUDA nvcc is: /usr/local/cuda-11.4/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda-11.4
-- Caffe2: Header version is: 11.4
-- Found cuDNN: v? (include: /usr/local/cuda-11.4/include, library: /usr/local/cuda-11.4/lib64/libcudnn.so)
CMake Error at cmake/public/cuda.cmake:158 (message):
PyTorch requires cuDNN 7 and above.
Call Stack (most recent call first):
cmake/Dependencies.cmake:896 (include)
CMakeLists.txt:373 (include)
-- Configuring incomplete, errors occurred!
See also "/home/assertst/pytorch/build/CMakeFiles/CMakeOutput.log".
See also "/home/assertst/pytorch/build/CMakeFiles/CMakeError.log".
FAILED: build.ninja
/home/assertst/anaconda3/bin/cmake --regenerate-during-build -S/home/assertst/pytorch -B/home/assertst/pytorch/build
ninja: error: rebuilding 'build.ninja': subcommand failed
Traceback (most recent call last):
File "setup.py", line 756, in <module>
build_deps()
File "setup.py", line 319, in build_deps
build_caffe2(version=version,
File "/home/assertst/pytorch/tools/build_pytorch_libs.py", line 59, in build_caffe2
cmake.build(my_env)
File "/home/assertst/pytorch/tools/setup_helpers/cmake.py", line 334, in build
self.run(build_args, my_env)
File "/home/assertst/pytorch/tools/setup_helpers/cmake.py", line 142, in run
check_call(command, cwd=self.build_dir, env=env)
File "/home/assertst/anaconda3/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '4']' returned non-zero exit status 1.
I don’t know why you are unable to remove a Python package, so I would recommend to create a new virtual environment.
The build fails to detect the cuDNN version properly. Since you were able to build it from source previously, what did you change that it fails now?
I didn’t change anything. The build is failing to detect CUDNN since I built it from source last time.
I thought that is happening because of Pytorch
Your GPU is too old and the PyTorch binaries are not building for compute capability 3.0 anymore (3.7 is the oldest supported architecture shipped in the binaries).
Wait… So you’re telling me that even if I wanted to build from source I can’t do it for a GPU with compute capability=3 anymore? Just confirming as I might have to run torch on a Nvidia Quadro K620
1 Like
import torch
device = torch.device(“cuda” if torch.cuda.is_available() else “cpu”)
print(f"Using device: {device}")
Using device: cuda
import torchCreate a tensor on the CPU
x = torch.randn(3, 3)
Move the tensor to the GPU
device = torch.device(“cuda” if torch.cuda.is_available() else “cpu”)
x = x.to(device)
F:\Python3\lib\site-packages\torch\cuda_init_.py:152: UserWarning:
Found GPU0 GeForce GT 730 which is of cuda capability 3.5.
PyTorch no longer supports this GPU because it is too old.
The minimum cuda capability supported by this library is 3.7.
warnings.warn(old_gpu_warn % (d, name, major, minor, min_arch // 10, min_arch % 10))
My question is Can I run 3 layer CNN or ResNet50 model ?