Pytorch is slower when with multiple threads ( on CPU)

1

I noticed that pytorch is slower when I set the number of threads to more than 1 (on cpu) with the following line of code:

torch.set_num_threads(30)

I was wondering if anyone has had the same issue?

2

@Rojin how many cores do your cpu have??
To know maximum number of threads that your pc can run simultaneously use this formula:-
Number of threads = number of cores * 2

3

The reason could be you use more number of threads(in your case, 30) than the actual number of threads that can be processed parallely by your machine.
Do

nproc

which gives the Core(s) per socket * Thread(s) per core
Pass that value to torch.set_num_threads().

You can also use torch.get_num_threads() to get the number of threads which will be used for parallelizing.
P.S: Not sure how to interpret the output given by torch.get_num_threads().

4

@mailcorahul @Skalwar

Thanks for your responses. I have 56 cores. which mean 112 threads. I set the number of threads to 112, and it defiantly slowed down the speed compared to using only 1 thread

5

I am facing the same dilemma, Have you got to know the reason behind this?

7

Firstly, you must confirm there are many threads which run on the different CPUs.

Then you should be attention to memory latency, if your machine is based on numa.