Is it required to set-up CUDA on PC before installing CUDA enabled pytorch?

10

i solved it. and yes you were right @albanD ! :smiley:
my nvidia drivers were old. i just updated the nvidia drivers by going to Start>Device Manager>Display adapters> select_ur_gpu >Right Click>Update Driver

Thanks a lot :slight_smile:

1 Like
11

Just curious, is the same true for cuDNN? I.e. does a user need to manually install cuDNN before CUDA enabled PyTorch will work or does the PyTorch installer do this for you?

12

The conda binaries and pip wheels ship also with the cudnn library so you don’t need to install it (same for NCCL).

2 Likes
13

Hello, I was following your discussion. I have updated GPU driver. Installed driver shows CUDA 11.2 .
While installing pytorch
conda install pytorch torchvision cudatoolkit=11.2 -c pytorch, it throws package not found error. Although i could install cudaroolkit=10.1 without error, I am still NOT able to use GPU with pyrorch. With the suggested [torch.rand(10).cuda()] I get [AssertionError: Torch not compiled with CUDA enabled] I donot understand where is the problem . I am wondering if I need update system path with for the cuda 10.1 ?

14

Hello albanD, I have updated GPU driver to the latest one 461.40. It shows the required CUDA version is 11.2. I am confused with the following.

  1. Should I need to install CUDA11.2 and set path accordingly before running conda pytorch torchvision…?
  2. Can I install any lower version of CUDA for the updated GPU driver?

thank you.

15

The conda binaries and pip wheels are not yet built with cudatoolkit=11.2 and you would have to use 9.2, 10.1, 10.2, or 11.0 as given in the install instructions.

Since these binaries ship with their own CUDA runtime, you would only need a local NVIDIA driver corresponding to the CUDA runtime you are selecting.
In case you want to build PyTorch from source or any custom CUDA extensions, you should install a matching local CUDA toolkit.

1 Like
16

Hi
I am not sure if this discussion is still valid.
I have a confusion whether in 2021 we still need to have CUDA toolkit installed in system before we install pytorch gpu version or does conda installs the toolkit as well by “conda install cudatoolkit=11.1” ?
And if conda installs the toolkit does pip3 also does that? even though toolkit is not explicitly mentioned in the following command:
“pip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html”

17

This wasn’t the case before and you would still only need to install the NVIDIA driver to run GPU workloads using the PyTorch binaries with the appropriately specified cudatoolkit version.
One limitation to this is that you would still need a locally installed CUDA toolkit to build custom CUDA extensions or PyTorch from source.

Yes, but the pip wheels are statically linking it instead of depending on the conda cudatoolkit.

2 Likes
18

I understood that we nvidia drivers.
I understood that cudnn and nvcc comes with pytorch installation.
I understood that cuda version that I specify should be supported by the nvidia driver.

What is still not 100% clear is:
Do we need to install Cuda toolkit separately or is it taken care by the pip3/conda ?

Lets ignore the part of building custom cuda extensions and pytorch from source.

2 Likes
19

No CUDA toolkit will be installed using the current binaries, but the CUDA runtime, which explains why you could execute GPU workloads, but not build anything from source.

2 Likes
20

Is it still true as of today (Oct 2021)? I am using torch 1.9. It looks like my torch installation using pip install comes with a CUDA version different from the one on nvidia-smi.

2 Likes
21

Its taking me to install pytorch 1.10 with CUDA 11.3 almost 8-9 hours? Is that ok? or something wrong? I am using GeForce RTX 3090

22

Something seems to be wrong.
Installing the pip wheels / conda binaries would take seconds or minutes depending on your internet connection while building from source could take between 15-120 mins on x86 depending on your workstation, the selected architectures etc., potentially much more on embedded devices if you build natively on them.
Could you describe your install approach and what kind of output you are seeing?

23

@ptrblck is there a way to avoid having pytorch install the CUDA runtime if I have everything installed on the system already, but still use pre-compiled binaries?
The sizes involved here are a bit insane to me: 1GB for pytorch conda package, almost 1GB for cuda conda package, and ~2GB for pytorch pip wheels.
Why do you force the CUDA package requirement on the CUDA-enabled pytorch conda package ?
I’d like to use pytorch in CI, but given the sizes involved here (and time for compilation from source), I’m not sure I want to use pytorch at all anymore

24

The binaries ship with the CUDA runtime for ease of use, as often users struggle to install the local CUDA toolkit with other libraries such as cuDNN and NCCL locally.
To use the GPU on your system in PyTorch you would thus only need to install the correct NVIDIA driver and one of the binary packages.

Building from source would use your local setup and would of course avoid having to download the binaries.

1 Like
25

I would be more interested in knowing if we need cudatoolkit=11.1:

conda install -y pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia

If CUDA is already installed?

26

Yes it’s needed, since the binaries ship with their own libraries and will not use your locally installed CUDA toolkit unless you build PyTorch from source or a custom CUDA extension.

2 Likes
27
  • Nvidia-smi: CUDA Version: 11.2
  • PyTorch install: CUDA 11.3 or 11.6?

Local machine

nvidia-smi
Tue Aug 30 11:38:20 2022
±----------------------------------------------------------------------------+
| NVIDIA-SMI 462.09 Driver Version: 462.09 CUDA Version: 11.2 |
|-------------------------------±---------------------±---------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 RTX A3000 Lapto… WDDM | 00000000:01:00.0 Off | N/A |
| N/A 51C P0 29W / N/A | 128MiB / 6144MiB | 0% Default |
| | | N/A |
±------------------------------±---------------------±---------------------+

PyTorch install

  • Start Locally | PyTorch currently offers CUDA 11.3 and 11.6.
  • Since the command nvidia-smi run on my local machine indicates CUDA Version 11.2, should I choose CUDA 11.3 or CUDA 11.6 when downloading PyTorch?

Thanks a lot for your time and any help…

28

I would recommend to use the latest one, i.e. 11.6 in your case.

1 Like
29

I appreciated all your all patient and elaborate answers about CUDA prerequisite. Sorry a quick question again:

If I have CUDA toolkit installed, compiled some CUDA extensions and uninstalled CUDA toolkit again. Do the CUDA extensions still work? I am not sure whether they dynamic link to CUDA tookit.

I asked such a weird question because I am using a docker image which does not include CUDA in advance and I have no permission to update this docker image.