Install pytorch with Cuda 12.1

This worked for me on Windows 10 with RTX A2000, with
CUDA 12.1,
cuDNN 8.8.1,
python 3.10 / anaconda
torch 2.0.1+cu118

1 Like

I get FALSE for: torch.cuda.is_available()

CUDA 12.1
torch: 2.0.1
torchvision: 0.15.2
python 3.10

Any suggestions?

Make sure the right PyTorch binary was installed by checking torch.version.cuda as well as the output of pip list | grep torch which should indicate the CUDA dependencies.

Thank you, i tried it.

torch.version.cuda:
I get “None”

pip list | findstr torch (I’m on Windows):
torch: 2.0.1
torchvision: 0.15.2

Any thoughts?

This would mean that you’ve installed the CPU-only binary and would need to select a proper CUDA dependency from the install matrix.

Thank you! I always used the CUDA 11.8 from the install matrix, but looks like it’s working now. If it helps others, here what I did:

I uninstalled torch through Termianl using “pip uninstall torch”

Then reinstalled using:
pip install torch torchvision --pre -f https://download.pytorch.org/whl/nightly/cu121/torch_nightly.html

This time it somehow installed correctly and now I get:

torch.cuda.is_available():
True

torch.version.cuda:
‘12.1’

pip list | findstr torch:
torch 2.1.0.dev20230604+cu121
torchvision 0.16.0.dev20230605+cu121

1 Like

What worked for me was:

pip uninstall torch
pip cache purge
pip install torch torchvision --pre -f https://download.pytorch.org/whl/nightly/cu121/torch_nightly.html
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

Cuda version with returned 12.1 with this command

nvidia-smi.exe
pip list | grep torch

returns this list

pytorch-lightning 1.6.4
torch 2.0.1+cu117
torchaudio 2.0.2
torchmetrics 0.9.2
torchvision 0.15.2+cu117

and

    import torch
    torch.cuda.is_available()

returns ‘True’

not sure if this is relevant, but i was running whisper and whisper was forcing cpu, now it works fine

2 Likes

Any updates on this issue ?

On what issue exactly? The CUDA 12.1 nightly binaries are available and install issues are caused by a broken environment in 99+% of cases. If you have trouble installing binaries, please give more information what fails and what you’ve tried.

HI, it did not worked for me. I am on Ubunt 20.04 and NVIDIA RTX3080. I am novice on linux. Can anyone help with step wise document on how to install pytorch with CUda activated?

Thanks,

Hi, Could you please guide me on this. I am using WSL2 on Windows11 and I prefer to install on Ubuntu. But on Powershell will also work for me. I have spent almost two weeks on this.Did multiple reinstallation of OS as well as NVidia cuda tool kit but of no success. I need to use my GPU for running my llama applications
Thnx,
Dilip

Hi, Could you please guide me on installing cuda enabled pytorch for windows powershell or on wsl2. I also made my laptop dual bootable with Ubuntu 20.04 But in all cases I failed to use pytorch with cuda enabled. I have spent almost two weeks on this.Did multiple reinstallation of OS as well as NVidia cuda tool kit but of no success. I need to use my GPU for running my llama applications
Thnx,
Dilip

You would need to install the NVIDIA drivers properly so that WSL2 can detect these (refer to any guide as this is unrelated to PyTorch). Once this is done, select the desired PyTorch setup from here, copy the install command, and paste it into your terminal.

Thanks for your quick response. I formated my laptop and reinstalled windows and wsl2. Now I can see the pytorch is detectingthe cuda. However, when I run the python program for Large language model. The program uses only CPU and GPU is not used.

This seems to be unrelated to the install process, so feel free to create a new topic describing the issue ideally with a minimal and executable code snippet showing the GPU is not used.

Hi, I wanted to know which cuda version of torchaudio is compatible with Pytorch cuda version 12.1
Can anyone help me with this?

The same stable and nightly releases should be compatible. Install torch and torchaudio together in the same install command for any release.

1 Like

Thank you for your advice.

Hlo, I am trying to install pytorch in my RTX 4090 GPU the problem is that I purged the cuda 12.3 version and installed the 11.7 version so that I can use pytorch >=1.8 but how much ever I try when I type nvidia-smi the same version is being shown the purge and reinstalling is unsuccessful . Actual the only thing I need is to install pytorch >= 1.8 can anyone tell me what I need to do

I don’t understand this workflow as the PyTorch binaries ship with their own CUDA runtime dependencies as explained in this topic previously. Your locally installed CUDA toolkit would be used if you build PyTorch from source or a custom CUDA extension. To execute PyTorch workloads you would need to install a supported NVIDIA driver.