RuntimeError: CUDA error: all CUDA-capable devices are busy or unavailable

It seemed that PyTorch failed to find my GPU. How can I fix this? Many thanks!

Python 3.8.3 (default, Jul 2 2020, 16:21:59)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import torch
torch.randn(5)
tensor([ 0.9021, 0.6406, 0.1212, 0.3215, -0.4576])

torch.randn(5).to(0)
Traceback (most recent call last):
File “”, line 1, in
RuntimeError: CUDA error: all CUDA-capable devices are busy or unavailable

Are you able to use other CUDA applications? If not, could you check if the compute mode of your GPUs is in an exclusive mode and reset it to the default one? Also, if you’ve upgraded something on your system make sure to reboot it afterwards.