I made a CppExtension, and the building had no problem, but when I import it from python, I’ve got an undefined symbol error:
Python 3.7.1 (default, Oct 23 2018, 17:15:52)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch_asr._latgen_lib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /home/jinserk/.pyenv/versions/3.7.1/lib/python3.7/site-packages/torch_asr/_latgen_lib.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN2at19UndefinedTensorImpl10_singletonE
I’m using gcc 8.2.0 on Ubuntu 18.10. Interestingly, I had no problem on the system of gcc 4.8.5 on CentOS7 with the exact same extension and setup.py.
I wonder whether the ATen shared libraries are automatically loaded by PyTorch or not. Do I need to add PyTorch directory to LD_LIBRARY_PATH manually? Obviously I didn’t do it on CentOS7. If I has to, why Ubuntu requires to do so while CentOS7 doesn’t?
Thanks for reply, @albanD !
Oh, I’m using torch_nightly on Ubuntu, but a custom built pytorch on CentOS. How to check the ABI compatibility between the pytorch binaries and the extension I made? If they don’t have the compatibility, how to fix it with the official build of torch_nightly?
So the ABI compatibility should be checked automatically and a warning raised if there is a risk of non-compatibility. I guess this check is faulty? @goldsborough will let you know for sure.
In the meantime, a temporary solution would be to try with a different gcc version. I am not sure which one is the best as it says 4.9+ in the warning but 8.2 might be too much?
Yes. I use Kaldi, a ASR framework, and link the shared libs from the project. However, when I put the lib directory into LD_LIBRARY_PATH, the error about the Kaldi has gone, but still it complains about the ATen library.