PyTorch installation with GPU support on Ubuntu

[rynczakd (Damian)]

While installing PyTorch with GPU support on Ubuntu (22.04 LTS), I ran into a few unknowns.

First of all, I checked that I have installed NVIDIA drivers using nvidia-smi command.
What I got as a result was a table in which I found: NVIDIA-SMI 535.154.05 / Driver Version: 535.154.05 / CUDA Version 12.2.
It shows that I have installed the drivers for the GPU.

Then, I checked that I have CUDA installed using nvcc --version command and I got: Build cuda_11.5.r11.5/compiler, so it also means that I have CUDA installed.

Now I want to install PyTorch using the instructions on the official site and here is the first question:

1. Which Compute Platform should I choose? On the official PyTorch site I can choose 11.8 and 12.1, but I have CUDA 11.5 installed. What is the relation between CUDA Version (12.2) obtained from nvidia-smi, CUDA 11.5 obtained from nvcc --version and CUDA version given in PyTorch installation website?

And I have few question about general PyTorch installation:

2. Installing PyTorch from the command below will make me have PyTorch installed along with the CUDA drivers? Can I use this command without having CUDA drivers installed before?

conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

3. Assuming I don’t have NVIDIA and CUDA drivers installed what steps should I follow before installing PyTorch? What are the correct steps to follow in general to install PyTorch correctly? Should I install NVIDIA GPU Drivers, then install cuDNN, then install CUDA Toolkt and then finally install PyTorch?

4. What we need cuDNN and CUDA Toolkit for in terms of using PyTorch?

5. What is the difference between installing PyTorch using conda and pip? Both can install drivers for my local computer?

6. After installing PyTorch I found that I have different version of CUDA in my computer. nvcc --version shows 11.5, but torch.version.cuda shows 12.1. What does this difference come from and what I can do with that issue? Does PyTorch use locally installed CUDA?

[ptrblck]

@rynczakd

Then, I checked that I have CUDA installed using nvcc --version command and I got: Build cuda_11.5.r11.5/compiler, so it also means that I have CUDA installed.

This won’t matter as PyTorch ships with its own CUDA runtime dependencies and your locally installed CUDA toolkit will be used if you are building PyTorch from source or a custom CUDA extension.

@rynczakd

Which Compute Platform should I choose?

I would recommend the latest CUDA toolkit (currently 12.1) as long as it’s compatible with your NVIDIA driver. Again, the locally CUDA toolkit doesn’t matter.

@rynczakd

Installing PyTorch from the command below will make me have PyTorch installed along with the CUDA drivers?

No, you need to install the NVIDIA driver yourself. The PyTorch binaries ship with CUDA runtime libraries, cuDNN, NCCL, etc.

@rynczakd

Can I use this command without having CUDA drivers installed before?

No, you need to install an NVIDIA driver, but don’t need a CUDA toolkit.

@rynczakd

Assuming I don’t have NVIDIA and CUDA drivers installed what steps should I follow before installing PyTorch?

Install an NVIDIA driver, then run pip install torch, which is enough to execute PyTorch workloads on your GPU.

I don’t respond to the other questions separately as they mainly target the locally installed CUDA toolkit which won’t be used if you install the binaries.