I am working with a paperspace vm with an A6000 GPU and have a transformers/pytorch model there.
I can’t get this to run on this GPU but it runs on the CPU.
PyTorch must have version 1.4.
The following is the setup of the VM and sample code.
Interestingly, the same model with the same settings (except nvcc, because no toolkit) runs fine on a P9000 from paperspace.
Anybody have an idea how I can get the model to run on the A6000 anyway?
import transformers, torch
print(transformers.__version__)
print(torch.__version__)
3.2.0
1.4.0+cu92
torch.cuda.is_available()
True
torch.cuda.device_count()
1
torch.cuda.current_device()
0
torch.cuda.device(0)
<torch.cuda.device at 0x7f679aac13d0>
torch.cuda.get_device_name(0)
'NVIDIA RTX A6000'
torch.version.cuda
'9.2'
nvidia-smi
Thu Aug 11 19:21:54 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01 Driver Version: 515.65.01 CUDA Version: 11.7 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA RTX A6000 On | 00000000:00:05.0 Off | Off |
| 30% 35C P8 18W / 300W | 5MiB / 49140MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1425 G /usr/lib/xorg/Xorg 4MiB |
+-----------------------------------------------------------------------------+
gcc --version
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
g++ --version
g++ (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.4 LTS
Release: 20.04
Codename: focal
x = torch.rand(5, 3, device="cpu")
print(x)
tensor([[0.8225, 0.6572, 0.9615],
[0.5285, 0.8547, 0.3622],
[0.8873, 0.2297, 0.5951],
[0.1628, 0.1683, 0.3286],
[0.5517, 0.0598, 0.7807]])
x = torch.rand(5, 3, device="cuda:0")
print(x)
# infinity-loop, does not terminate, no error message