`num_threads` in `torch.utils.benchmark` only for CPU?

[ptrblck]

From the docs:

torch.get_num_threads() → int
Returns the number of threads used for parallelizing CPU operations

So yes, the threads setting refers to CPU-only workloads.

@OverLordGoldDragon

and does benchmark use all GPU threads?

The benchmark util. won’t specify or change the device and your code is responsible to use the GPU or CPU.