Pytorch not detecting NVIDIA RTX 3080Ti and CUDA 11.6 due to sm_86 compatibility issue

Hello.

I installed torch as per getting started locally guide. I have a dual xeon system with RTX 3080Ti and CUDA 11.6, CUDNN8.4.1 … They are all working fine. However with torch I get error

NVIDIA GeForce RTX 3080 Ti with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3080 Ti GPU with PyTorch, please check the instructions at Start Locally | PyTorch

I have already followed the instructions but there is no success.

The whole steps are as under:

metanet@metanet-Precision-Tower-7810:~/ProgramFiles/speechbrain/recipes$ pip3 install torch torchvision torchaudio --extra-index-url
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https//pypiorg/simple, https//pypingcnvidiacom, https//downloadpytorchorg/whl/cu116
Requirement already satisfied: torch in /home/metanet/.local/lib/python3.8/site-packages (1.11.0)
Requirement already satisfied: torchvision in /home/metanet/.local/lib/python3.8/site-packages (0.12.0)
Requirement already satisfied: torchaudio in /home/metanet/.local/lib/python3.8/site-packages (0.11.0)
Requirement already satisfied: typing-extensions in /home/metanet/.local/lib/python3.8/site-packages (from torch) (4.2.0)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /home/metanet/.local/lib/python3.8/site-packages (from torchvision) (9.1.1)
Requirement already satisfied: numpy in /home/metanet/.local/lib/python3.8/site-packages (from torchvision) (1.21.6)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from torchvision) (2.22.0)

metanet@metanet-Precision-Tower-7810:~/ProgramFiles/speechbrain/recipes$ nvidia-smi
Sat Jul 2 11:24:12 2022
±----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03 Driver Version: 510.47.03 CUDA Version: 11.6 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce … On | 00000000:03:00.0 On | N/A |
| 30% 45C P8 32W / 350W | 494MiB / 12288MiB | 13% Default |
| | | N/A |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 2298 G /usr/lib/xorg/Xorg 190MiB |
| 0 N/A N/A 2713 G /usr/bin/gnome-shell 51MiB |
| 0 N/A N/A 3746 G …257163255691216877,131072 81MiB |
| 0 N/A N/A 23276 G …veSuggestionsOnlyOnDemand 92MiB |
| 0 N/A N/A 24491 G …b/thunderbird/thunderbird 74MiB |
±----------------------------------------------------------------------------+

**metanet@metanet-Precision-Tower-7810:~/ProgramFiles/speechbrain/recipes$ cat check-device.py **
import torch

torch.cuda.is_available()
torch.cuda.current_device()
torch.cuda.device(0)
torch.cuda.device_count()
torch.cuda.get_device_name(0)

device = torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’)
print(‘Using device:’, device)
print()

if device.type == ‘cuda’:
print(torch.cuda.get_device_name(0))
print(‘Memory Usage:’)
print(‘Allocated:’, round(torch.cuda.memory_allocated(0)/10243,1), ‘GB’)
print('Cached: ', round(torch.cuda.memory_reserved(0)/1024
3,1), ‘GB’)

$ python3 check-device.py
/home/metanet/.local/lib/python3.8/site-packages/torch/cuda/init.py:145: UserWarning:
NVIDIA GeForce RTX 3080 Ti with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3080 Ti GPU with PyTorch, please check the instructions at Start Locally | PyTorch

warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
Using device: cuda

NVIDIA GeForce RTX 3080 Ti
Memory Usage:
Allocated: 0.0 GB
Cached: 0.0 GB

This error:

The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.

would mean that the installed binary isn’t shipping with sm_86 and is thus most likely using the CUDA10.2 runtime while you would need CUDA11.
Your pip install command doesn’t install anything as all requirements are already satisfied, so uninstall PyToch via pip uninstall torch -y and conda uninstall pytorch -y until no packages can be found and install the right one.

I had the same error message when executing print(torch.cuda.get_device_name(0)).

I tried different CUDA versions: 11.0, 11.3, 11.6, and 11.7 for an Nvidia 3080 GPU but all had the same issue. In then end going from Python3.6 to Python3.7 solved the issue.

Most likely you did not install the right PyTorch binary with this CUDA runtime, but might have tried to install the CUDA toolkit manually?
E.g. PyTorch with the CUDA 11.6 runtime works on my 3090 and Python 3.6 is already EOL and was already deprecated in mid. January.