Pytorch finds cuda despite nvcc not found?

Pytorch sees cuda and runs well on GPU, but nvcc appears to be not found

import torch
torch.cuda.is_available()  # True
torch.cuda.device_count()   #1
torch.cuda.current_device()  # 0
torch.cuda.get_device_name(0) # NVIDIA GeForce RTX 3090
$ nvidia-smi     
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 68%   72C    P2   341W / 420W |   9876MiB / 24576MiB |    100%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A    234092      C   python3                          9871MiB |
+-----------------------------------------------------------------------------+
$nvcc --version
Command 'nvcc' not found, but can be installed with:
sudo apt install nvidia-cuda-toolkit

Once I run $sudo apt install nvidia-cuda-toolkit, nvidia-smi gets removed and pytorch can’t recongize GPU. To install nvidia-smi, I run $sudo apt install nvidia-utils-515-server, but meanwhile nvcc get uninstalled. This looks like a chicken egg problem.

$sudo updatedb; locate nvcc
/etc/nvcc.profile
~/.local/lib/python3.10/site-packages/torch/share/cmake/Caffe2/Modules_CUDA_fix/upstream/FindCUDA/run_nvcc.cmak
...

OS: Ubuntu server 22.04

The PyTorch binaries ship with their own CUDA runtime (as well as cuDNN, NCCL etc.) and don’t need a locally installed CUDA toolkit to execute code but only a properly installed NVIDIA driver.
Your local CUDA toolkit (with the compiler) will be used if you build PyTorch from source or a custom CUDA extension.
Based on your described issue, I guess your CUDA toolkit installation uninstalled the NVIDIA driver as well and/or broke it, so try a new full install and make sure CUDA applications work again.

1 Like