Cuda not available for RTX 6000 mobile

windows
21

From the install instructions:

# CUDA 11.7
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2

installs the wheel properly:

pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2
Collecting torch==2.0.1
  Downloading torch-2.0.1-cp310-cp310-manylinux1_x86_64.whl.metadata (24 kB)
Collecting torchvision==0.15.2
  Downloading torchvision-0.15.2-cp310-cp310-manylinux1_x86_64.whl.metadata (11 kB)
Collecting torchaudio==2.0.2
  Downloading torchaudio-2.0.2-cp310-cp310-manylinux1_x86_64.whl.metadata (1.2 kB)
...
Collecting nvidia-cuda-nvrtc-cu11==11.7.99 (from torch==2.0.1)
  Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-runtime-cu11==11.7.99 (from torch==2.0.1)
  Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cuda-cupti-cu11==11.7.101 (from torch==2.0.1)
  Downloading nvidia_cuda_cupti_cu11-11.7.101-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cudnn-cu11==8.5.0.96 (from torch==2.0.1)
...
Successfully installed MarkupSafe-2.1.5 certifi-2024.2.2 charset-normalizer-3.3.2 cmake-3.29.2 filelock-3.14.0 idna-3.7 jinja2-3.1.4 lit-18.1.4 mpmath-1.3.0 networkx-3.3 numpy-1.26.4 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-cupti-cu11-11.7.101 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.2.10.91 nvidia-cusolver-cu11-11.4.0.1 nvidia-cusparse-cu11-11.7.4.91 nvidia-nccl-cu11-2.14.3 nvidia-nvtx-cu11-11.7.91 pillow-10.3.0 requests-2.31.0 sympy-1.12 torch-2.0.1 torchaudio-2.0.2 torchvision-0.15.2 triton-2.0.0 typing-extensions-4.11.0 urllib3-2.2.1
...
python -c "import torch; print(torch.__version__); print(torch.version.cuda)"
2.0.1+cu117
11.7
1 Like
22

I just tried that and it returns:
ERROR: Could not find a version that satisfies the requirement torch==2.0.1 (from versions: 2.2.0, 2.2.1, 2.2.2, 2.3.0)
ERROR: No matching distribution found for torch=2.0.1

23

In this case your Python version might be wrong (either too new or too old). I was using Python3.10 as seen in the file names.

1 Like
24

Oh ok I will downgrade then I was using 3.12.3

25

that worked. RVC still says it cant find any GPU I guess I have to see in their code how it’s checking for that. it’s quite annoying.

26

After installing pytorce 2.0.1 and python 3.10 now the same code in OP returns this:

CUDA Available: False
Traceback (most recent call last):
File “…\Desktop\import torch.py”, line 3, in
print("CUDA Device Name: “, torch.cuda.get_device_name(0))
File “…\LocalCache\local-packages\Python310\site-packages\torch\cuda_init_.py”, line 365, in get_device_name
return get_device_properties(device).name
File “…\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\torch\cuda_init_.py”, line 395, in get_device_properties
_lazy_init() # will define get_device_properties
File "…\LocalCache\local-packages\Python310\site-packages\torch\cuda_init.py”, line 239, in _lazy_init
raise AssertionError(“Torch not compiled with CUDA enabled”)
AssertionError: Torch not compiled with CUDA enabled

27

I checked the code inside RVC they are using the same code to see if cuda is available. So I just went back to install:
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

and then cuda 12.1 again because installing other ones seem to change the default cuda, but still same error as “AssertionError: Torch not compiled with CUDA enabled”

You guys said cuda binaries ship with pytorch so that means you don’t really need to install cuda from nvidia website at all?

The only difference now is that I don’t use python 3.12.3 but python 3.10.

28

Uninstalling all pytorch and components and reinstalling I get different error. So just installing with new pip commands do not do the right things? We have to uninstall each time to be sure?

In any case I get the shm error which I thought was fixed:

“local-packages\Python310\site-packages\torch\lib\shm.dll” or one of its dependencies."

I used:

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

29

The shm issue was reported for 2.3.0 as seen in the linked issue so you can either install the fixed nightly or apply a workaround suggested in the GitHub issue for your current installation.

1 Like
30

Thanks a lot that works. Can you confirm an uninstall is necessary for it to update the files correctly? Because if I didn’t uninstall all pytorch stuff, and just call the new nightly pip install command, i still get the same error. but after uninstall and reinstall it works.

pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu121

So now I used this command:
python -c “import torch; print(torch.version)”

and got this error:

Error loading “…/site-packages\torch\lib\fbgemm.dll” or one of its dependencies."

Could this be another bug like shm.dll thing?

31

This is what I always do. Users have reported that “updating” PyTorch might work, but I’m always uninstalling it before installing a new stable or nightly release as it’ll pull all dependencies anyway.

I’m not familiar with the fbgemm issue but the linked post seems to suggest installing VS fixes it.

1 Like