Cuda not available for RTX 6000 mobile

windows
1

Hi,

I have mobile RTX 6000 on my laptop:

When I run this code, it returns False:
import torch
print(torch.cuda.is_available())

And this returns “Torch not compiled with CUDA enabled”
print(torch.cuda.get_device_name(0))

I am using torch from RVC, but I don’t remember if I installed it myself for another project like OpenAI whisper variants.

Mon May 6 11:37:19 2024
±----------------------------------------------------------------------------------------+
| NVIDIA-SMI 551.23 Driver Version: 551.23 CUDA Version: 12.4 |
|-----------------------------------------±-----------------------±---------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 Quadro RTX 6000 WDDM | 00000000:01:00.0 On | N/A |
| N/A 66C P2 58W / 132W | 5957MiB / 24576MiB | 3% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+

What should I do to fix this?

2

Hi @orionflame,

When you installed PyTorch did you install PyTorch with CUDA? It seems you may have installed the cpu only version of PyTorch, as torch.cuda.is_available() returns False.

Make sure to select an appropriate CUDA version for the PyTorch install via the getting-started page, here: Start Locally | PyTorch

These other threads on the forum might help you debug any issues:

  1. Torch.cuda.is_available() returns False even CUDA is installed
  2. Torch.cuda.is_available() is false CUDA Version: 11.4
3

Thanks I just installed cuda 12.1 and nvcc reports that I have it. But when I run the same code now I get this:

The specified module could not be found. Error loading local-packages\Python312\site-packages\torch\lib\shm.dll" or one of its dependencies.

This is after running this install:
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

I will try to see how to fix it.

4

If you already have torch installed, you might have to update the existing install (as well as installing the CUDA add-on) rather than just using the install command directly from the getting-started page, but I’m not 100% sure.

Perhaps it might be best to create a new environment (which has CUDA support from the get go?) and try the install command from the getting-started page?

If not, @ptrblck might be able to help! (apologies for the ping!)

5

thx but pip install updates the relevant components based on the command no? I thought it updated it to support cuda 12.1 based on the command I posted.

what’s the cuda addon? i installed it from nvidia first.

also what do you mean by new env? like conda? i just wanted to use RVC

6

I’m just trying to figure out if I might be missing something obvious (which I most definitely am).

The issue seems to be pytorch fails loading a particular .dll file, so perhaps a fresh install in a clean environment might be the best initial solution (before @ptrblck hopefully responds). Or perhaps you could force reinstall the pip3 command you stated above?

7

I just read on this thread: Error loading "\lib\site-packages\torch\lib\shm.dll" or one of its dependencie - #2 by Alexander_DE (which you’re also on). The error might be due to using pytorch2.3 (which was released quite recently).

Perhaps do a clean install of PyTorch 2.2 and see if the error is resolved?

8

Is there a way to specify the version? When I click previous versions on the official website, it shows conda commands. The same GUI for forming the pip command is not available so not sure if I can specify the version for all or I have to figure out individual versions for each module.

9

You should be able to find install commands for previous install version here: Previous PyTorch Versions | PyTorch

For CUDA 12.1 install, the command is:
pip install torch==2.2.2 torchvision==0.17.2 torchaudio==2.2.2 --index-url https://download.pytorch.org/whl/cu121

1 Like
10

Thanks that seems to work, I get this printed now:

CUDA Available: True
CUDA Device Name: Quadro RTX 6000

11

Now I need to figure out why RVC still says " Unfortunately there is no compatible GPU available to support your training." I imagine they dont support pytorch 2.2.

12

Thanks for the ping @AlphaBetaGamma96.
The issue is tracked here and might come from an MKL update on Windows:

Aaaand I found the solution. The MKL package, when installed in user context, installs the libraries that shm.dll is linked to into %APPDATA%\Python\Library\Bin - which per default isn’t in the PATH env var. Adding that path to the PATH env var allows for the DLL loader to actually find the MKL dlls

The topic also suggests workarounds @orionflame.

1 Like
13

If i am not wrong this issue is fixed right?

I am trying to use this:

But it will says it can’t find any supported GPU.

On the page that say this:

  1. If you are using Nvidia Ampere architecture (RTX30xx) in Windows, according to the experience of #21, you need to specify the cuda version corresponding to pytorch.

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

If I am not wrong my GPU is Turing so I shouldn’t be using the latest pytorch?

I asked there but they didn’t reply anyway for a week.

Any ideas? :pray:

14

If you’re trying to install CUDA 11.7, perhaps try PyTorch 2.0.1 via the install page?

1 Like
15

thx i will try this now.

can multiple cuda versions exist together? i assume i dont have to uninstall the other one.

16

Turing GPUs are still supported in the latest CUDA releases and all of our current binaries with CUDA 11.8, 12.1, and 12.4 support these.

PyTorch binaries ship with their own CUDA dependencies and your locally installed CUDA toolkit won’t be used unless you build PyTorch from source or a custom CUDA extension.
You cannot install multiple PyTorch binaries into one environment.

1 Like
17

Oh I see, I thought i had to install cuda separately for pytorch to work. Then I will just install PyTorch 2.0.1 as Alpha recommended.

18

I am trying to install it for 11.7 but I got an error saying no matching distribution found for torch==2.0.0

# CUDA 11.7
pip install torch==2.0.0 torchvision==0.15.1 torchaudio==2.0.1
# CUDA 11.8
pip install torch==2.0.0 torchvision==0.15.1 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118

from the help above: Previous PyTorch Versions | PyTorch

also shouldnt 11.7 command have this?

–index-url https://download.pytorch.org/whl/cu117

It’s a bit confusing.

19

Shouldn’t it be torch==2.0.1?
The command on the website is:
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.7 -c pytorch -c nvidia

1 Like
20

Yes I wanted to use that but I dont have conda that’s why I was looking for comparable one to get cuda 11.7.

I dont know how to convert that condo command into regular pip. Because in pip it’s using torch name and pytorch name so not sure why they are different like that.