Pytorch says GPU busy/unavailable even though it returns 'True' with " cuda.is_available() "

1

Hi all!

I have been trying to make Pytorch work with NVIDIA A100 - 80GB GPU .
Following is the nvidia-smi output :

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.203.03   Driver Version: 450.203.03   CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  A100 80GB PCIe      On   | 00000000:01:01.0 Off |                   On |
| N/A   30C    P0    42W / 300W |      0MiB / 81252MiB |     N/A      Default |
|                               |                      |              Enabled |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| MIG devices:                                                                |
+------------------+----------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |         Memory-Usage |        Vol|         Shared        |
|      ID  ID  Dev |           BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
|                  |                      |        ECC|                       |
|==================+======================+===========+=======================|
|  No MIG devices found                                                       |
+-----------------------------------------------------------------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

I have installed Pytorch using the command available on official website :

pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113

which gives me " torch==1.12.1+cu113 " version of Pytorch. Upon checking if Pytorch detects GPU or not using torch.cuda.is_available() I get True as return.
While a basic Tensor torch.Tensor([1,2,3]).to("cuda") gives me the following output

image
image945×100 4.71 KB

I have multiple combinations of CUDA Drivers - Pytorch versions but nothing works.

Can anyone please help?

2

Try to disable MIG and rerun your script.

3

Yes, I figured that out eventually. Thanks for the response though!