Threading of Model Pytorch Android

IvanKobzarev · December 2, 2019, 9:44pm

Hello @mohit7
At the moment we do not have this setting, but we are thinking to add this option to control it.
Current threading model is determined by device, the number of threads is about number of BIG cores on the device.
More details about number of threads per device you can find in the code of function caffe2::ThreadPool::defaultThreadPool()

github.com

pytorch/pytorch/blob/master/caffe2/utils/threadpool/ThreadPool.cc#L24




// Whether or not threadpool caps apply to iOS
C10_DEFINE_int(caffe2_threadpool_ios_cap, true, "");


namespace caffe2 {


// Default smallest amount of work that will be partitioned between
// multiple threads; the runtime value is configurable
constexpr size_t kDefaultMinWorkSize = 1;


std::unique_ptr<ThreadPool> ThreadPool::defaultThreadPool() {
  CAFFE_ENFORCE(cpuinfo_initialize(), "cpuinfo initialization failed");
  int numThreads = cpuinfo_get_processors_count();


  bool applyCap = false;
#if C10_ANDROID
  applyCap = FLAGS_caffe2_threadpool_android_cap;
#elif C10_IOS
  applyCap = FLAGS_caffe2_threadpool_ios_cap;
#endif

Please write us if you have threadding issues with some particular device.