Install pytorch with Cuda 12.1

1

hello,
I have a GPU Nvidia GTX 1650 with Cuda 12.1.
I want to install the pytorch with Cuda, but the latest version is Cuda 11.8 on the website.
Is it possible to install version 11.8 and I have 12.1?

1 Like
2

Yes, the PyTorch binaries ship with their own CUDA runtime, cuDNN, NCCL etc. and will work with newer drivers. Your local CUDA toolkit will be used if you build PyTorch from source or a custom CUDA extension.

3 Likes
3

Torch built with cuda1.18 works under cuda12.1

I compiled a new version with cuda12.1.
There seems to be no performance advantage

2 Likes
4

I banged my head for a couple of days trying to get PyTorch (2.0.1+cu118) working with cuda12.1 in Unbuntu 20.04.

  • Installing PyTorch via conda did not work. torch.cuda.is_available() returned False
  • Compiling PyTorch did not work (for me). torch.cuda.is_available() returned False
  • Installing PyTorch via PIP worked. This returns:
    • torch.cuda.is_available() is True
    • torch.backends.cudnn.version() is 8700
    • torch.backends.cudnn.enabled is True
2 Likes
5

Could you please be more specific? What is the exact command that you have used?

1 Like
6

No performance advantage compared to what? Also, where can you learn about your specific compilation process?

7

This is my shell setup script:

conda_forge_packages="pypdf2 screeninfo opencv matplotlib scipy pandas openpyxl statsmodels"
conda update -n base -c defaults conda
conda create -n my_env -c conda-forge python=3.10 ${conda_forge_packages}
source ~/miniconda3/etc/profile.d/conda.sh
conda activate my_env
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
1 Like
8
  1. Latest pytorch built with cuda11.8
  2. Official pytorch document
9

Then there must be something I’m doing wrong because for me, it is not compatible. I’m getting the error:

RuntimeError:
      The detected CUDA version (12.1) mismatches the version that was used to compile
      PyTorch (11.8). Please make sure to use the same CUDA versions.

I have the nvidia drivers 530.30.02 with CUDA 12.1. I’m currently following the installation options of NerfStudio.

Their install instructions are:
pip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 --extra-index-url https://download.pytorch.org/whl/cu118

And the error happens when I try to install
pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

10

The error is raised if you are trying to build a third-party module from source, which will use your locally installed CUDA toolkit. Either downgrade your local CUDA toolkit to 11.8 or update your PyTorch binaries to 12.1 to use matching versions.

1 Like
11

This worked for me on Windows 10 with RTX A2000, with
CUDA 12.1,
cuDNN 8.8.1,
python 3.10 / anaconda
torch 2.0.1+cu118

1 Like
12

I get FALSE for: torch.cuda.is_available()

CUDA 12.1
torch: 2.0.1
torchvision: 0.15.2
python 3.10

Any suggestions?

13

Make sure the right PyTorch binary was installed by checking torch.version.cuda as well as the output of pip list | grep torch which should indicate the CUDA dependencies.

14

Thank you, i tried it.

torch.version.cuda:
I get “None”

pip list | findstr torch (I’m on Windows):
torch: 2.0.1
torchvision: 0.15.2

Any thoughts?

15

This would mean that you’ve installed the CPU-only binary and would need to select a proper CUDA dependency from the install matrix.

16

Thank you! I always used the CUDA 11.8 from the install matrix, but looks like it’s working now. If it helps others, here what I did:

I uninstalled torch through Termianl using “pip uninstall torch”

Then reinstalled using:
pip install torch torchvision --pre -f https://download.pytorch.org/whl/nightly/cu121/torch_nightly.html

This time it somehow installed correctly and now I get:

torch.cuda.is_available():
True

torch.version.cuda:
‘12.1’

pip list | findstr torch:
torch 2.1.0.dev20230604+cu121
torchvision 0.16.0.dev20230605+cu121

1 Like
17

What worked for me was:

pip uninstall torch
pip cache purge
pip install torch torchvision --pre -f https://download.pytorch.org/whl/nightly/cu121/torch_nightly.html
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

Cuda version with returned 12.1 with this command

nvidia-smi.exe

pip list | grep torch

returns this list

pytorch-lightning 1.6.4
torch 2.0.1+cu117
torchaudio 2.0.2
torchmetrics 0.9.2
torchvision 0.15.2+cu117

and

    import torch
    torch.cuda.is_available()

returns ‘True’

not sure if this is relevant, but i was running whisper and whisper was forcing cpu, now it works fine

2 Likes
18

Any updates on this issue ?

19

On what issue exactly? The CUDA 12.1 nightly binaries are available and install issues are caused by a broken environment in 99+% of cases. If you have trouble installing binaries, please give more information what fails and what you’ve tried.

20

HI, it did not worked for me. I am on Ubunt 20.04 and NVIDIA RTX3080. I am novice on linux. Can anyone help with step wise document on how to install pytorch with CUda activated?

Thanks,