Should I install the extra cudatoolkit and cudnn?

windows
1

I install the latest pytorch from the official site with the command “conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia”. When I run the code “torch.cuda.is_available()”, the output is True. Does it mean that I don’t have to install the cudatoolkit and cudnn if I wanna run my model on GPU ? My computer is brand new and I don’t install the cudatoolkit and cudnn additionally.

1 Like
2

The PyTorch binaries ship with all CUDA runtime dependencies and you don’t need to locally install a CUDA toolkit or cuDNN. Only a properly installed NVIDIA driver is needed to execute PyTorch workloads on the GPU.

3

Seriously? I know I should trust what you say because you are an expert, but your opinion contradicts other Google search results and other posts here on the forum.

1 Like
4

You don’t need to trust me of course and can stick to other people’s opinions. If you check my posts here I’ve verified multiple times that no CUDA toolkit is needed and the binaries will work fine with their own shipped dependencies e.g. by installing them in a clean Ubuntu docker container. I’m also updating the CUDA dependencies in the binaries in the pytorch/builder status and should thus know how dependencies are used.

1 Like
5

Shoud NVIDIA driver be installed on host machine or in container?

6

The driver needs to be installed on the host. The container can ship with a newer driver in addition, if you want to use CUDA’s forward compatibility for DataCenter GPUs.

2 Likes
7

I think that the cudatoolkit do not need to be installed by pip, because the command is similar as pip install torch==1.10.1+cu111, where the cudatoolkit is installed with pytorch.
While we use conda to install torch, such as conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=10.2 -c pytorch will install cudatoolkit.
But I am confused about conda install pytorch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 -c pytorch, which seems like do not need cudatoolkit anymore.

8

Thank you for the clarification regarding the CUDA toolkit! I have a follow-up question: when using vLLM, Hugging Face, or Lightning with pytorch+cu121, is installing them in a clean Ubuntu Docker image similarly sufficient?