How to check GPU usage when running pytorch

1

Dear expert,
How can I check my GPU usage properly? I tried to use the taks manager,but it says GPU usage is 0%, like below:

image
image1526×1080 50.5 KB

However, if I use the navidia-smi, it seems GPU usage is 94% (I am not sure what this 94% is):
image
image1736×890 30.6 KB

I can see the GPU memery was consumed and I have all indications from pytorch that I am using GPU(such as torch.cuda.is_available(), and torch.cuda.get_device_name(0) ). But why GPU is alway 0% in task manager? Also I can’t see CUDA from dropdown in GPU tab:

image
image1055×712 22.7 KB

I have CUDA version 12.0, and i have below installed:

torch.version
‘2.0.1+cu118’
torch.version.cuda
‘11.8’
I also did another check. I set the device=torch.device(‘cpu’) to make a comparison, and I found out CPU usage is rising (CPU usage is normal when I use GPU).
Seems something is wrong (because it feels the training process using GPU is vvvvvery long), but I can’t be sure. Could you please advise?

Thanks,
Long

2

I’m not sure if it’s different in Windows. On Linux, nvidia-smi only shows you to current snapshot. If you want to continuously observe the usage while running something, try

watch -n 0.5 nvidia-smi

This updates the data every 0.5 seconds.

3

Thanks for you answer. Does the 94% in my screenshot represent the GPU usage? Do you have any idea why the task manager in Windows shows differently?

4

Yes, 94% refers to the GPU utilization. Sorry, I on Linux (Ubuntu) and cannot help with Windows.

5

Many thanks for your reply.

6

For both Linux and Windows you may use nvitop. Install it by pip install nvitop and it will show more details at a glance.

1 Like
7

Thanks Lucian, it works like a charm.