How to building static libtorch on ubuntu 18.04 without CUDA

How to build libtorch static library without CUDA

For some reason, I need to use the static library of libtorch, e.g. libtorch.a (not libtorch.so as provided by the official website". So I would like to compile it from source. My compute is Ubuntu 18.04 with CUDA, but I set NO_CUDA=1.

Here is what I tried:
git clone https://github.com/pytorch/pytorch.git

cd pytorch
mkdir build_libtorch && cd build_libtorch
NO_CUDA=1
BUILD_SHARED_LIBS=OFF
python ../tools/build_libtorch.py
 

However, the compilation failed with the error:

Determining if the pthread_create exist failed with the following output:
Change Dir: /home/yinzheng/repos/pytorch/build_libtorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/home/yinzheng/anaconda3/bin/ninja cmTC_aa6e6
[1/2] Building C object CMakeFiles/cmTC_aa6e6.dir/CheckSymbolExists.c.o
[2/2] Linking C executable cmTC_aa6e6
FAILED: cmTC_aa6e6
: && /usr/bin/cc -rdynamic CMakeFiles/cmTC_aa6e6.dir/CheckSymbolExists.c.o -o cmTC_aa6e6 && :
CMakeFiles/cmTC_aa6e6.dir/CheckSymbolExists.c.o: In function main': CheckSymbolExists.c:(.text+0x1b): undefined reference to pthread_create’
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

File /home/yinzheng/repos/pytorch/build_libtorch/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c:
/* */
#include <pthread.h>

int main(int argc, char** argv)
{
(void)argv;
#ifndef pthread_create
return ((int*)(&pthread_create))[argc];
#else
(void)argc;
return 0;
#endif
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /home/yinzheng/repos/pytorch/build_libtorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/home/yinzheng/anaconda3/bin/ninja cmTC_c5dad
[1/2] Building C object CMakeFiles/cmTC_c5dad.dir/CheckFunctionExists.c.o
[2/2] Linking C executable cmTC_c5dad
FAILED: cmTC_c5dad
: && /usr/bin/cc -DCHECK_FUNCTION_EXISTS=pthread_create -rdynamic CMakeFiles/cmTC_c5dad.dir/CheckFunctionExists.c.o -o cmTC_c5dad -lpthreads && :
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

Performing C SOURCE FILE Test C_HAS_AVX_1 failed with the following output:
Change Dir: /home/yinzheng/repos/pytorch/build_libtorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/home/yinzheng/anaconda3/bin/ninja cmTC_e13be
[1/2] Building C object CMakeFiles/cmTC_e13be.dir/src.c.o
FAILED: CMakeFiles/cmTC_e13be.dir/src.c.o
/usr/bin/cc -fopenmp -DC_HAS_AVX_1 -fPIE -o CMakeFiles/cmTC_e13be.dir/src.c.o -c src.c
src.c: In function ‘main’:
src.c:7:7: warning: AVX vector return without AVX enabled changes the ABI [-Wpsabi]
a = _mm256_set1_ps(0);
^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/7/include/immintrin.h:41:0,
from src.c:2:
/usr/lib/gcc/x86_64-linux-gnu/7/include/avxintrin.h:1310:1: error: inlining failed in call to always_inline ‘_mm256_set1_ps’: target specific option mismatch
_mm256_set1_ps (float __A)
^~~~~~~~~~~~~~
src.c:7:7: note: called from here
a = _mm256_set1_ps(0);
^~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.

Source file was:

#include <immintrin.h>

int main()
{
__m256 a;
a = _mm256_set1_ps(0);
return 0;
}

Performing C SOURCE FILE Test C_HAS_AVX2_1 failed with the following output:
Change Dir: /home/yinzheng/repos/pytorch/build_libtorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/home/yinzheng/anaconda3/bin/ninja cmTC_e89cc
[1/2] Building C object CMakeFiles/cmTC_e89cc.dir/src.c.o
FAILED: CMakeFiles/cmTC_e89cc.dir/src.c.o
/usr/bin/cc -fopenmp -DC_HAS_AVX2_1 -fPIE -o CMakeFiles/cmTC_e89cc.dir/src.c.o -c src.c
src.c: In function ‘main’:
src.c:7:7: warning: AVX vector return without AVX enabled changes the ABI [-Wpsabi]
a = _mm256_abs_epi16(a);
^~~~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/7/include/immintrin.h:43:0,
from src.c:2:
/usr/lib/gcc/x86_64-linux-gnu/7/include/avx2intrin.h:63:1: error: inlining failed in call to always_inline ‘_mm256_abs_epi16’: target specific option mismatch
_mm256_abs_epi16 (__m256i __A)
^~~~~~~~~~~~~~~~
src.c:7:7: note: called from here
a = _mm256_abs_epi16(a);
^~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.

Source file was:

#include <immintrin.h>

int main()
{
__m256i a = {0};
a = _mm256_abs_epi16(a);
__m256i x;
_mm256_extract_epi64(x, 0); // we rely on this in our AVX2 code
return 0;
}

Performing C SOURCE FILE Test CXX_HAS_AVX_1 failed with the following output:
Change Dir: /home/yinzheng/repos/pytorch/build_libtorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/home/yinzheng/anaconda3/bin/ninja cmTC_a61ef
[1/2] Building C object CMakeFiles/cmTC_a61ef.dir/src.c.o
FAILED: CMakeFiles/cmTC_a61ef.dir/src.c.o
/usr/bin/cc -fopenmp -DCXX_HAS_AVX_1 -fPIE -o CMakeFiles/cmTC_a61ef.dir/src.c.o -c src.c
src.c: In function ‘main’:
src.c:7:7: warning: AVX vector return without AVX enabled changes the ABI [-Wpsabi]
a = _mm256_set1_ps(0);
^~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/7/include/immintrin.h:41:0,
from src.c:2:
/usr/lib/gcc/x86_64-linux-gnu/7/include/avxintrin.h:1310:1: error: inlining failed in call to always_inline ‘_mm256_set1_ps’: target specific option mismatch
_mm256_set1_ps (float __A)
^~~~~~~~~~~~~~
src.c:7:7: note: called from here
a = _mm256_set1_ps(0);
^~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.

Source file was:

#include <immintrin.h>

int main()
{
__m256 a;
a = _mm256_set1_ps(0);
return 0;
}

Performing C SOURCE FILE Test CXX_HAS_AVX2_1 failed with the following output:
Change Dir: /home/yinzheng/repos/pytorch/build_libtorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/home/yinzheng/anaconda3/bin/ninja cmTC_f2d12
[1/2] Building C object CMakeFiles/cmTC_f2d12.dir/src.c.o
FAILED: CMakeFiles/cmTC_f2d12.dir/src.c.o
/usr/bin/cc -fopenmp -DCXX_HAS_AVX2_1 -fPIE -o CMakeFiles/cmTC_f2d12.dir/src.c.o -c src.c
src.c: In function ‘main’:
src.c:7:7: warning: AVX vector return without AVX enabled changes the ABI [-Wpsabi]
a = _mm256_abs_epi16(a);
^~~~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/7/include/immintrin.h:43:0,
from src.c:2:
/usr/lib/gcc/x86_64-linux-gnu/7/include/avx2intrin.h:63:1: error: inlining failed in call to always_inline ‘_mm256_abs_epi16’: target specific option mismatch
_mm256_abs_epi16 (__m256i __A)
^~~~~~~~~~~~~~~~
src.c:7:7: note: called from here
a = _mm256_abs_epi16(a);
^~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.

Source file was:

#include <immintrin.h>

int main()
{
__m256i a = {0};
a = _mm256_abs_epi16(a);
__m256i x;
_mm256_extract_epi64(x, 0); // we rely on this in our AVX2 code
return 0;
}

But I am sure that I have pthreads

(base) yinzheng@yinzheng-LC0:/lib/x86_64-linux-gnu$ ll *thread*
-rwxr-xr-x 1 root root 144976 4月  17  2018 libpthread-2.27.so*
lrwxrwxrwx 1 root root     18 8月  30 19:13 libpthread.so.0 -> libpthread-2.27.so*
-rw-r--r-- 1 root root  35648 4月  17  2018 libthread_db-1.0.so
lrwxrwxrwx 1 root root     19 8月  30 19:13 libthread_db.so.1 -> libthread_db-1.0.so

and I add /lib/x86_64-linux-gnu to LD_LIBRARY_PATH

(base) yinzheng@yinzheng-LC0:/lib/x86_64-linux-gnu$ echo $LD_LIBRARY_PATH 
/home/yinzheng/anaconda3/lib:/lib:/usr/lib:

So is there any suggestion and help to tell me how to compile libtorch static library with CUDA on ubuntu?
Thanks very much in advance!

Regards
Yin

2 Likes

Same here… Di you find the solution? Thank you …