Building from Source : Parallel Compile and Optimization Questions

1

Hi Devs,

I am building from source and had a few questions.

  1. MAX_JOBS env var doesn’t seem to have any effect on parallel compile. htop shows activity on <10% of cores throughout build process.

  2. I am using following command to inject “-O3” flag in setup.py parameters. (My objective is to unroll and vectorize loops for a 64-core/256HT CPU.)

sed -i ‘s/'-Wno-write-strings'/'-Wno-write-strings' , '-O3'/’ setup.py

Is there an environment variable I can use to specify optimization flags? I can submit a pull request to add such a flag if you’d like because sed method is brittle.

  1. MKL libraries are in conda3/pkgs/mkl-2018.0.0-hb491cac_4/lib/ folder, but not being detected during build. Does MKL detection rely on CMAKE_LIBRARY_PATH env var, or something else?

Thanks!

2
  1. by default all cores are used for compile
  2. O3 is already used for default compiles.
  3. it relies on CMAKE_LIBRARY_PATH or CMAKE_PREFIX_PATH. Usually this works out export CMAKE_PREFIX_PATH="$(dirname $(which conda))/../"
1 Like
3

Thank you Soumith :v: