Install pytorch with CUDA 11

22

Hi @ptrblck ,

I have installed pytorch via cmd in windows 10 with CUDA 12.1

  • I have NVIDIA driver version 537.70
  • I have installed CUDA 12.1.0
  • Pytorch version: 2.1.0+cu121

But when I run the DeviceQuery.exe i get the result as False, CUDA is also not getting detected.

Nvidia

The scenario is I am building a Document QA system using LLM, I have to load the model onto the GPU, but I am not able to do.

What might be the reason for this hindrance?

Note : I have Nvidia A40-12Q with 12GB and this is VDI (Virtual Desktop Infrastructure) based system.

23

I don’t know what’s causing the issue, but you would have to make sure your system is able to detect and use the GPU before you execute any PyTorch workloads.