Hello. I’m trying to install PyTorch v1.4.0 from source and am experiencing the following error:
[4070/4411] Building CXX object test_api/CMakeFiles/test_api.dir/modules.cpp.o
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "setup.py", line 743, in <module>
build_deps()
File "setup.py", line 316, in build_deps
cmake=cmake)
File "/home/seokwon/pytorch/tools/build_pytorch_libs.py", line 62, in build_caffe2
cmake.build(my_env)
File "/home/seokwon/pytorch/tools/setup_helpers/cmake.py", line 339, in build
self.run(build_args, my_env)
File "/home/seokwon/pytorch/tools/setup_helpers/cmake.py", line 141, in run
check_call(command, cwd=self.build_dir, env=env)
File "/home/seokwon/anaconda3/envs/seanconda/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '20']' returned non-zero exit status 1.
I followed all of the steps accordingly specified in the GitHub README file and also made sure that CUDA version was appropriate. Does anybody know how to solve this issue? I’ve tried looking elsewhere online but was unsuccessful. Thanks in advance!
Sorry, but how do I check if I’m running out of memory?
Also on a side note, the server I’m using has two versions of CUDA installed: 9.0 and 10.1. In the /usr/local/ directory there are both cuda-10.1/ and cuda-9.0/, and the cuda file is pointing to /usr/local/cuda-9.0/ (I’m not sure if this is correct terminology). I’m trying to use CUDA 10.1, would this be the cause of the problem? If I install PyTorch using conda install pytorch torchvision cudatoolkit=10.1 -c pytorch then it install fine, but torch.cuda.is_available() returns False.
So I managed to solve my issue, but I haven’t been able to solve this error per se.
I wasn’t aware that the CUDA driver didn’t match CUDA. I assumed that my server administrator took care of it but perhaps there was an error in the process. Regardless, installing PyTorch with the method specified on PyTorch’s homepage now works.
Good to hear you figured out the (binary?) installation issue.
However, that doesn’t solve the build from source failure.
Could you clean the build (python setup.py clean), update all submodules (git submodule update --init --recursive) and rebuild it?
While the build is running, you could use htop to observe the system RAM.
I made a new virtual environment and did as you suggested, but it doesn’t seem like it’s a memory problem. There’s plenty of memory left even when the process is taking place.
I got the same errors when installing pytorch from source for versions 1.6 and 1.7 on Ubuntu 18.04. I installed the nvidia drivers, cuda 10.2 and cudnn 8.0.3 for 2080ti gpus. I followed the instructions to install pytorch from source and still got the ninja error from above also at “random” points of the installation. I didn’t have these problems on my local machine with Ubuntu 20.04 with the same GPU and same versions. Any help is much appreciated.
The previous error was created by a potentially old version of GLIBCXX:
../../lib/libc10.so: error: undefined reference to 'std::thread::_State::~_State()', version 'GLIBCXX_3.4.22'
../../lib/libc10.so: error: undefined reference to 'typeinfo for std::thread::_State', version 'GLIBCXX_3.4.22'
../../lib/libc10.so: error: undefined reference to 'std::runtime_error::runtime_error(char const*)', version 'GLIBCXX_3.4.21'
Could you check the install log for errors?
Note that the installation might continue for some steps and the error might be shown in previous lines.
I’ve proposed a solution, it will be great if people can go ahead and do the following changes.
Add #include in the following files
pytorch/third_party/tensorpipe/tensorpipe/common/allocator.h
pytorch/third_party/tensorpipe/tensorpipe/common/memory.h
pytorch/third_party/gloo/gloo/types.h
Note this is on Fedora 42 + CPU my final build successful output is
~/pytorch main !1 python setup.py develop 1 err torch py 11:59:04
Building wheel torch-2.8.0a0+git5616fa4
-- Building version 2.8.0a0+git5616fa4
-- Checkout nccl release tag: v2.26.5-1
cmake --build . --target install --config Release
[1/2] Install the project...
-- Install configuration: "Release"
running develop
/usr/lib/python3.13/site-packages/setuptools/command/develop.py:41: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` and ``easy_install``.
Instead, use pypa/build, pypa/installer or other
standards-based tools.
See https://github.com/pypa/setuptools/issues/917 for details.
********************************************************************************
!!
easy_install.initialize_options(self)
/usr/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!
********************************************************************************
Please avoid running ``setup.py`` directly.
Instead, use pypa/build, pypa/installer or other
standards-based tools.
See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
********************************************************************************
!!
self.initialize_options()
running egg_info
writing torch.egg-info/PKG-INFO
writing dependency_links to torch.egg-info/dependency_links.txt
writing entry points to torch.egg-info/entry_points.txt
writing requirements to torch.egg-info/requires.txt
writing top-level names to torch.egg-info/top_level.txt
reading manifest file 'torch.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.*' under directory 'ios'
warning: no files found matching '*.*' under directory 'modules'
warning: no previously-included files matching '*.o' found anywhere in distribution
warning: no previously-included files matching '*.dylib' found anywhere in distribution
warning: no previously-included files matching '*.swp' found anywhere in distribution
adding license file 'LICENSE'
adding license file 'NOTICE'
writing manifest file 'torch.egg-info/SOURCES.txt'
running build_ext
-- Building with NumPy bindings
-- Not using cuDNN
-- Not using CUDA
-- Not using XPU
-- Using MKLDNN
-- Not using Compute Library for the Arm architecture with MKLDNN
-- Not using CBLAS in MKLDNN
-- Not using NCCL
-- Building with distributed package:
-- USE_TENSORPIPE=True
-- USE_GLOO=True
-- USE_MPI=False
-- Building Executorch
-- Using ITT
Copying functorch._C from functorch/functorch.so to /home/sumantro/pytorch/build/lib.linux-x86_64-cpython-313/functorch/_C.cpython-313-x86_64-linux-gnu.so
copying functorch/functorch.so -> /home/sumantro/pytorch/build/lib.linux-x86_64-cpython-313/functorch/_C.cpython-313-x86_64-linux-gnu.so
building 'torch._C' extension
creating build/temp.linux-x86_64-cpython-313
creating build/temp.linux-x86_64-cpython-313/torch
creating build/temp.linux-x86_64-cpython-313/torch/csrc
gcc -fno-strict-overflow -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -fexceptions -fcf-protection -fexceptions -fcf-protection -fexceptions -fcf-protection -O3 -fPIC -I/usr/include/python3.13 -c torch/csrc/stub.c -o build/temp.linux-x86_64-cpython-313/torch/csrc/stub.o -Wall -Wextra -Wno-strict-overflow -Wno-unused-parameter -Wno-missing-field-initializers -Wno-unknown-pragmas -fno-strict-aliasing
creating build/lib.linux-x86_64-cpython-313/torch
gcc -shared build/temp.linux-x86_64-cpython-313/torch/csrc/stub.o -L/home/sumantro/pytorch/torch/lib -L/usr/lib64 -ltorch_python -o build/lib.linux-x86_64-cpython-313/torch/_C.cpython-313-x86_64-linux-gnu.so -Wl,-rpath,$ORIGIN/lib
copying build/lib.linux-x86_64-cpython-313/torch/_C.cpython-313-x86_64-linux-gnu.so -> torch
Creating /home/sumantro/.local/lib/python3.13/site-packages/torch.egg-link (link to .)
Adding torch 2.8.0a0+git5616fa4 to easy-install.pth file
Installing torchfrtrace script to /home/sumantro/.local/bin
Installing torchrun script to /home/sumantro/.local/bin
Installed /home/sumantro/pytorch
Processing dependencies for torch==2.8.0a0+git5616fa4
Searching for setuptools==74.1.3
Best match: setuptools 74.1.3
Adding setuptools 74.1.3 to easy-install.pth file
Using /usr/lib/python3.13/site-packages
Searching for fsspec==2024.6.1
Best match: fsspec 2024.6.1
Adding fsspec 2024.6.1 to easy-install.pth file
Using /home/sumantro/.local/lib/python3.13/site-packages
Searching for jinja2==3.1.6
Best match: jinja2 3.1.6
Adding jinja2 3.1.6 to easy-install.pth file
Using /usr/lib/python3.13/site-packages
Searching for networkx==3.3
Best match: networkx 3.3
Adding networkx 3.3 to easy-install.pth file
Using /home/sumantro/.local/lib/python3.13/site-packages
Searching for sympy==1.14.0
Best match: sympy 1.14.0
Adding sympy 1.14.0 to easy-install.pth file
Installing isympy script to /home/sumantro/.local/bin
Using /home/sumantro/.local/lib/python3.13/site-packages
Searching for typing-extensions==4.12.2
Best match: typing-extensions 4.12.2
Adding typing-extensions 4.12.2 to easy-install.pth file
Using /usr/lib/python3.13/site-packages/setuptools/_vendor
Searching for filelock==3.15.4
Best match: filelock 3.15.4
Adding filelock 3.15.4 to easy-install.pth file
Using /usr/lib/python3.13/site-packages
Searching for MarkupSafe==3.0.2
Best match: MarkupSafe 3.0.2
Adding MarkupSafe 3.0.2 to easy-install.pth file
Using /usr/lib64/python3.13/site-packages
Searching for mpmath==1.3.0
Best match: mpmath 1.3.0
Adding mpmath 1.3.0 to easy-install.pth file
Using /home/sumantro/.local/lib/python3.13/site-packages
Finished processing dependencies for torch==2.8.0a0+git5616fa4
