Claude 2 says this is an ABI incompatibility issue between RANSAC voting and PyTorch however I am not sure how to resolve it.
Here is details also cross-posted in original repo:
(hybridpose) mona@ada:~/HybridPose$ cd lib/ransac_voting_gpu_layer
(hybridpose) mona@ada:~/HybridPose/lib/ransac_voting_gpu_layer$ python setup.py build_ext --inplace
running build_ext
/home/mona/anaconda3/envs/hybridpose/lib/python3.10/site-packages/torch/utils/cpp_extension.py:502: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
warnings.warn(msg.format('we could not find ninja.'))
/home/mona/anaconda3/envs/hybridpose/lib/python3.10/site-packages/torch/utils/cpp_extension.py:424: UserWarning: There are no g++ version bounds defined for CUDA version 11.8
warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
(hybridpose) mona@ada:~/HybridPose$ cd lib/regressor
(hybridpose) mona@ada:~/HybridPose/lib/regressor$ make
g++ -O3 --std=c++11 -shared -Ieigen -IDataStructure -IOperation -ILinearAlgebra -o wrapper.so -fPIC wrapper.cpp DataStructure/*.cpp Operation/*.cpp LinearAlgebra/*.cpp
(hybridpose) mona@ada:~/HybridPose$ LD_LIBRARY_PATH=lib/regressor:$LD_LIBRARY_PATH python src/train_core.py
Traceback (most recent call last):
File "/home/mona/HybridPose/src/train_core.py", line 16, in <module>
from trainers.coretrainer import CoreTrainer
File "/home/mona/HybridPose/./trainers/coretrainer.py", line 10, in <module>
from lib.ransac_voting_gpu_layer.ransac_voting_gpu import ransac_voting_layer_v3
File "/home/mona/HybridPose/lib/ransac_voting_gpu_layer/ransac_voting_gpu.py", line 2, in <module>
import lib.ransac_voting_gpu_layer.ransac_voting as ransac_voting
ImportError: /home/mona/HybridPose/lib/ransac_voting_gpu_layer/ransac_voting.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c106detail19maybe_wrap_dim_slowEllb
I have:
(hybridpose) mona@ada:~/HybridPose$ conda env export > environment.yml
(hybridpose) mona@ada:~/HybridPose$ cat environment.yml
name: hybridpose
channels:
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- _openmp_mutex=5.1=1_gnu
- bzip2=1.0.8=h7b6447c_0
- ca-certificates=2023.08.22=h06a4308_0
- ld_impl_linux-64=2.38=h1181459_1
- libffi=3.4.4=h6a678d5_0
- libgcc-ng=11.2.0=h1234567_1
- libgomp=11.2.0=h1234567_1
- libstdcxx-ng=11.2.0=h1234567_1
- libuuid=1.41.5=h5eee18b_0
- ncurses=6.4=h6a678d5_0
- openssl=3.0.11=h7f8727e_2
- pip=23.2.1=py310h06a4308_0
- python=3.10.13=h955ad1f_0
- readline=8.2=h5eee18b_0
- setuptools=68.0.0=py310h06a4308_0
- sqlite=3.41.2=h5eee18b_0
- tk=8.6.12=h1ccaba5_0
- tzdata=2023c=h04d1e81_0
- wheel=0.41.2=py310h06a4308_0
- xz=5.4.2=h5eee18b_0
- zlib=1.2.13=h5eee18b_0
- pip:
- astropy==5.3.4
- certifi==2023.7.22
- charset-normalizer==3.3.0
- click==8.1.7
- cloudpickle==3.0.0
- contourpy==1.1.1
- cycler==0.12.1
- dask==2023.10.0
- fonttools==4.43.1
- fsspec==2023.9.2
- idna==3.4
- imageio==2.31.5
- importlib-metadata==6.8.0
- joblib==1.3.2
- kiwisolver==1.4.5
- lazy-loader==0.3
- locket==1.0.0
- markupsafe==2.1.3
- matplotlib==3.8.0
- mpmath==1.3.0
- networkx==3.1
- numpy==1.26.1
- opencv-python==4.8.1.78
- packaging==23.2
- partd==1.4.1
- pillow==10.1.0
- platformdirs==3.11.0
- pooch==1.7.0
- pyamg==5.0.1
- pyerfa==2.0.1
- pyparsing==3.1.1
- python-dateutil==2.8.2
- pywavelets==1.4.1
- pyyaml==6.0.1
- requests==2.31.0
- scikit-image==0.22.0
- scikit-learn==1.3.1
- scipy==1.11.3
- simpleitk==2.3.0
- six==1.16.0
- sympy==1.12
- threadpoolctl==3.2.0
- tifffile==2023.9.26
- toolz==0.12.0
- torch==2.1.1+cu118
- torchaudio==2.1.1+cu118
- torchtext==0.14.0
- torchvision==0.16.1+cu118
- tqdm==4.66.1
- typing-extensions==4.8.0
- urllib3==2.0.6
- zipp==3.17.0
prefix: /home/mona/anaconda3/envs/hybridpose
(hybridpose) mona@ada:~/HybridPose$ python
Python 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] on linux
and
(base) mona@ada:~$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
and
(base) mona@ada:~$ nvidia-smi
Wed Dec 13 09:41:35 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.104.12 Driver Version: 535.104.12 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA RTX 6000 Ada Gene... On | 00000000:52:00.0 On | Off |
| 31% 60C P2 72W / 300W | 5067MiB / 49140MiB | 3% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 2310 G /usr/lib/xorg/Xorg 1088MiB |
| 0 N/A N/A 2535 G /usr/bin/gnome-shell 102MiB |
| 0 N/A N/A 3040 G ...AAAAAAAACAAAAAAAAAA= --shared-files 41MiB |
| 0 N/A N/A 3678 G ...irefox/3358/usr/lib/firefox/firefox 308MiB |
| 0 N/A N/A 26577 G /usr/lib/thunderbird/thunderbird 19MiB |
| 0 N/A N/A 151053 G ...AAAAAAAIAAAAAAAAAA== --shared-files 102MiB |
| 0 N/A N/A 388240 G ...97542783,9996950370839883190,262144 81MiB |
| 0 N/A N/A 862586 C python 918MiB |
| 0 N/A N/A 863180 C python 730MiB |
| 0 N/A N/A 864122 C python 730MiB |
| 0 N/A N/A 901572 C+G ./blender 379MiB |
| 0 N/A N/A 1014271 G ...sion,SpareRendererForSitePerProcess 55MiB |
| 0 N/A N/A 1023738 G meshlab 13MiB |
| 0 N/A N/A 1024568 C+G ./blender 357MiB |
+---------------------------------------------------------------------------------------+
Checked the issues, the solution here didn’t work for my case Import Error: Ransac Voting Layer - undefined symbo · Issue #18 · chensong1995/HybridPose · GitHub
Original author of RANSAC voting mentioned add import torch
above the line which already is there compile ransac_voting · Issue #13 · zju3dv/pvnet · GitHub
import torch
import lib.ransac_voting_gpu_layer.ransac_voting as ransac_voting
I also have:
(hybridpose) mona@ada:~/HybridPose$ ldd lib/ransac_voting_gpu_layer/ransac_voting.cpython-310-x86_64-linux-gnu.so
linux-vdso.so.1 (0x00007fffe68ec000)
libc10.so => not found
libtorch_cpu.so => not found
libtorch_python.so => not found
libcudart.so.11.0 => /usr/local/cuda-11.8/lib64/libcudart.so.11.0 (0x00007fd6e6c00000)
libc10_cuda.so => not found
libstdc++.so.6 => /home/mona/anaconda3/envs/hybridpose/lib/libstdc++.so.6 (0x00007fd6e6800000)
libgcc_s.so.1 => /home/mona/anaconda3/envs/hybridpose/lib/libgcc_s.so.1 (0x00007fd6e6fca000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd6e6400000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd6e7070000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd6e6fc5000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd6e6fbe000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fd6e6fb9000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd6e6ed2000)
and
(hybridpose) mona@ada:~/HybridPose$ find . -type d | grep lib
./lib
./lib/datasets
./lib/datasets/__pycache__
./lib/__pycache__
./lib/regressor
./lib/regressor/DataStructure
./lib/regressor/LinearAlgebra
./lib/regressor/eigen
./lib/regressor/__pycache__
./lib/regressor/Operation
./lib/ransac_voting_gpu_layer
./lib/ransac_voting_gpu_layer/__pycache__
./lib/ransac_voting_gpu_layer/build
./lib/ransac_voting_gpu_layer/build/temp.linux-x86_64-cpython-310
./lib/ransac_voting_gpu_layer/build/temp.linux-x86_64-cpython-310/src
./lib/ransac_voting_gpu_layer/build/lib.linux-x86_64-cpython-310
./lib/ransac_voting_gpu_layer/src
I tried giving the pytorch path but still the ldd doesn’t find torch
(hybridpose) mona@ada:~/HybridPose/lib/ransac_voting_gpu_layer$ python setup.py clean
running clean
(hybridpose) mona@ada:~/HybridPose/lib/ransac_voting_gpu_layer$ python setup.py build_ext --inplace -L /home/mona/anaconda3/envs/hybridpose/lib/python3.10/site-packages/torch/lib
running build_ext
/home/mona/anaconda3/envs/hybridpose/lib/python3.10/site-packages/torch/utils/cpp_extension.py:502: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
warnings.warn(msg.format('we could not find ninja.'))
/home/mona/anaconda3/envs/hybridpose/lib/python3.10/site-packages/torch/utils/cpp_extension.py:424: UserWarning: There are no g++ version bounds defined for CUDA version 11.8
warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
(hybridpose) mona@ada:~/HybridPose/lib/ransac_voting_gpu_layer$ cd ../..
(hybridpose) mona@ada:~/HybridPose$ ldd lib/ransac_voting_gpu_layer/ransac_voting.cpython-310-x86_64-linux-gnu.so
linux-vdso.so.1 (0x00007fff1f7f0000)
libc10.so => not found
libtorch_cpu.so => not found
libtorch_python.so => not found
libcudart.so.11.0 => /usr/local/cuda-11.8/lib64/libcudart.so.11.0 (0x00007fbaac600000)
libc10_cuda.so => not found
libstdc++.so.6 => /home/mona/anaconda3/envs/hybridpose/lib/libstdc++.so.6 (0x00007fbaac200000)
libgcc_s.so.1 => /home/mona/anaconda3/envs/hybridpose/lib/libgcc_s.so.1 (0x00007fbaac9e0000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbaabe00000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbaaca86000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbaac9db000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbaac9d4000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fbaac9cf000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbaac8e8000)
(hybridpose) mona@ada:~/HybridPose$ LD_LIBRARY_PATH=lib/regressor:$LD_LIBRARY_PATH python src/train_core.py
Traceback (most recent call last):
File "/home/mona/HybridPose/src/train_core.py", line 16, in <module>
from trainers.coretrainer import CoreTrainer
File "/home/mona/HybridPose/./trainers/coretrainer.py", line 10, in <module>
from lib.ransac_voting_gpu_layer.ransac_voting_gpu import ransac_voting_layer_v3
File "/home/mona/HybridPose/lib/ransac_voting_gpu_layer/ransac_voting_gpu.py", line 2, in <module>
import lib.ransac_voting_gpu_layer.ransac_voting as ransac_voting
ImportError: /home/mona/HybridPose/lib/ransac_voting_gpu_layer/ransac_voting.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c106detail19maybe_wrap_dim_slowEllb
Also, tried with gcc-10 no help
(hybridpose) mona@ada:~/HybridPose/lib/ransac_voting_gpu_layer$ CC=gcc-10 CXX=g++-10 python setup.py build_ext --inplace
running build_ext
/home/mona/anaconda3/envs/hybridpose/lib/python3.10/site-packages/torch/utils/cpp_extension.py:502: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
warnings.warn(msg.format('we could not find ninja.'))
/home/mona/anaconda3/envs/hybridpose/lib/python3.10/site-packages/torch/utils/cpp_extension.py:424: UserWarning: There are no g++-10 version bounds defined for CUDA version 11.8
warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')