RTX 5090 CUDA initialization fails in PyTorch Docker containers (works with RTX 4090)

1

I’m experiencing CUDA initialization failures with RTX 5090 GPUs in PyTorch Docker containers, while the same setup works perfectly with RTX 4090.

Hardware & Software:

  • GPU: NVIDIA GeForce RTX 5090 (8x cards)

  • OS: Ubuntu 22.04 (Jammy)

  • Kernel: 6.8.0-79-generic

  • NVIDIA Drivers tested: 570.x.x, 575.64.03, 580.95.05

  • CUDA versions: 12.8, 13.0 (based on driver)

What works:

  • nvidia-smi shows all 8 GPUs correctly

  • Basic CUDA containers work fine (nvidia/cuda:12.1.0-base-ubuntu22.04)

  • Direct CUDA C API initialization succeeds (cuInit() works)

  • Same setup works perfectly with RTX 4090 cards

What fails:

  • PyTorch CUDA initialization in Docker containers

  • Tested images: pytorch/pytorch:2.6.0-cuda12.6-cudnn9-devel, pytorch/pytorch:2.4.0-cuda12.1-cudnn9-runtime, pytorch/pytorch:2.0.1-cuda11.7-cudnn8-runtime

  • Error: CUDA initialization: CUDA driver initialization failed, you might not have a CUDA gpu

  • PyTorch reports: torch.cuda.is_available() = False but torch.cuda.device_count() = 8

Questions:

  1. Which PyTorch nightly build should I use for RTX 5090 support?

  2. Is there a specific CUDA version requirement?

  3. Any recommended Docker images or installation methods?

Any guidance would be greatly appreciated!

2

Test code:

$ docker run --rm --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=0   pytorch/pytorch:2.4.0-cuda12.1-cudnn9-runtime   python -c “import torch; print(f’CUDA available: {torch.cuda.is_available()}‘); print(f’GPU count: {torch.cuda.device_count()}’)”
/opt/conda/lib/python3.11/site-packages/torch/cuda/**init**.py:128: UserWarning: CUDA initialization: CUDA driver initialization failed, you might not have a CUDA gpu. (Triggered internally at /opt/conda/conda-bld/pytorch_1720538456841/work/c10/cuda/CUDAFunctions.cpp:108.)
return torch.\_C.\_cuda_getDeviceCount() > 0
CUDA available: False
GPU count: 1
3

@ptrblck
Sorry for the highlight, but I have see you quite active on this topic