Pytorch multithread testing is slower than one thread(How to improve performance for testing)

I have trained a pytorch model and write one test py script to predict(input one image and load model and predict the result).
I have 10 images, there are two ways:

  1. one thread
    just loop the 10 images order by order, the time of every test result is about 40ms
  2. multithread
    use python multithread, but the time of every test result is about 500ms, but the total execute time is similar with way one.

It seems the multithread is not working.

In the test script, the num_workers param of load testset method(torch.utils.data.DataLoader) is set to 0 is faster then 2 or 10.(it seems the bigger of num_workers ,the slower of the result)

Could you help me how to improve the test performance?

Many thanks,
Alex

Can you try setting env var OMP_NUM_THREADS=1 before running your python script?

I am sorry,it does not work for me .