How to check GPU usage when running pytorch

Dear expert,
How can I check my GPU usage properly? I tried to use the taks manager,but it says GPU usage is 0%, like below:


However, if I use the navidia-smi, it seems GPU usage is 94% (I am not sure what this 94% is):

I can see the GPU memery was consumed and I have all indications from pytorch that I am using GPU(such as torch.cuda.is_available(), and torch.cuda.get_device_name(0) ). But why GPU is alway 0% in task manager? Also I can’t see CUDA from dropdown in GPU tab:

I have CUDA version 12.0, and i have below installed:

torch.version
‘2.0.1+cu118’
torch.version.cuda
‘11.8’
I also did another check. I set the device=torch.device(‘cpu’) to make a comparison, and I found out CPU usage is rising (CPU usage is normal when I use GPU).
Seems something is wrong (because it feels the training process using GPU is vvvvvery long), but I can’t be sure. Could you please advise?

Thanks,
Long

I’m not sure if it’s different in Windows. On Linux, nvidia-smi only shows you to current snapshot. If you want to continuously observe the usage while running something, try

watch -n 0.5 nvidia-smi

This updates the data every 0.5 seconds.

Thanks for you answer. Does the 94% in my screenshot represent the GPU usage? Do you have any idea why the task manager in Windows shows differently?

Yes, 94% refers to the GPU utilization. Sorry, I on Linux (Ubuntu) and cannot help with Windows.

Many thanks for your reply.

For both Linux and Windows you may use nvitop. Install it by pip install nvitop and it will show more details at a glance.

1 Like

Thanks Lucian, it works like a charm.