GPU is not available

1

I am establishing a model with pytorch and trying to use GPUs to test my model. However, there is an error occuring with the following warning:
"
/home/usrname/softwares/anaconda3/envs/torch12/lib/python3.7/site-packages/torch/cuda/init.py:83: UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at …/c10/cuda/CUDAFunctions.cpp:109.)
return torch._C._cuda_getDeviceCount() > 0
"
python: 3.7.13
pytorch: 1.12.0+cu102 (pip)
cuda: 10.2
GPU: NVIDIA GeForce RTX 2080Ti
GPU Driver Version: 440.100
OS: Centos 7
The GPU is working normally which is able to display graphical interface of the server. The output of nvidia-smi:
image

It is available last month with nothing changed except that we moved the servers to another room. Did someone meet similar problems? Could you please give some advice to solve the problem? Thanks a lot !!!

2

Check if any other CUDA application is still executable and if not, reinstall the drivers as it seems your workstation is running into a setup issue.

3

Thank you for your suggestion. I am sure that no other CUDA application is working and will try to reinstall the driver.