Is CUDA 12.0 supported with any pytorch version?

1

If you look at this page, there are commands how to install a variety of pytorch versions given the CUDA version. However, the only CUDA 12 version seems to be 12.1. My cluster machine, for which I do not have admin right to install something different, has CUDA 12.0.

I tried to modify one of the lines like:

conda install pytorch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 pytorch-cuda=12.0 -c pytorch -c nvidia

But this did not work. Is there a way to install pytorch with CUDA 12.0? If so, is there documentation on how to go about that? Thanks!

2

Yes, the current PyTorch code base supports all CUDA 12 toolkit versions if you build from source. The install matrix on the website shows the prebuilt binaries which ship with their own CUDA runtime dependencies. If you install these your locally installed CUDA toolkit won’t be used. You would only need to properly install an NVIDIA driver.

3

Thank you @ptrblck. So are you saying I need to build from source against CUDA 12.0? Or are you saying I can type something like:

pip3 install torch torchvision torchaudio

And the torch binary installed with ignore CUDA 12.0 on the system and use it’s own internal CUDA 12.1 libraries? Or are you saying both. :slight_smile:

4

For posterity, using

pip3 install torch torchvision torchaudio

did work assuming I had the CUDA 12.0 driver installed as verified by the nvidia-smi command. The only issue is I had to update my environment variables for this one library for whatever reason:

export LD_LIBRARY_PATH=$HOME/.local/lib/python3.10/site-packages/nvidia/nvjitlink/lib:$LD_LIBRARY_PATH
5

Yes, this is the case. If you have a local CUDA toolkit installed and are seeing library conflicts, removing the locally installed CUDA toolkit from the LD_LIBRARY_PATH can be used as a workaround.

Good to hear it’s working now! :slight_smile:

6

Hello Sir,
My Cuda Version is 12.5 and Driver version is 555.85 . There is no available compatible pytorch version for this. There is a trouble when running this nvidia/tts_en_fastpitch. What needs to be done to sort out this issue?
Thanks.

7

My previous post explains the support: Is CUDA 12.0 supported with any pytorch version? - #2 by ptrblck

8

Is it right to interpret the fact that “the torch binary installed [will] ignore CUDA 12.0 on the system and use it’s own internal CUDA 12.1 libraries” as meaning that CUDA is statically linked to the PyTorch install, or am I thinking about this all wrong?

9

No, only the CUDA runtime is statically linked to PyTorch as is the common approach. Other CUDA dependencies will be installed as dependencies as pip packages and will be dynamically opened during the runtime.

10

This finally worked for me! After grappling with grok for hours… Thanks!

11

You mean, even 2.10.0 can also be build && install by source code based on CUDA12.0? An other question, now latest pytorch version build file(setting.py) current force need AVX or AVX2? I have an old cpu amd FX8350(only support AVX not AVX2) and an old GPU cmp 50hx(20G) to run vllm 14b q8, whether I need to set any special configuration to build&&install latest pytorch version on CUDA12.0?