Pytorch CUDA. Why it's loading CPU?

1

Why the code is loading my CPU at 17% ? I thought the code in “with” block is executing on the GPU.

device = torch.device("cuda:0")

size = (50, 50)

x = torch.rand(size, dtype=torch.float32, device=device)

y = torch.rand(size, dtype=torch.float32, device=device)

with torch.cuda.device(device):
    torch.cuda.synchronize()
    for _ in range(1000000):
        result = torch.add(x, y)

There was no such problem if “for” iterations is much more less but tensors are much more bigger.

GPU RTX 2070

2

Your Python code is still being executed on the CPU, which will dispatch to the CUDA kernel and launch it, so a CPU utilization is expected.

1 Like