[SOLVED] Building from source keeps failing (Ubuntu 18.04.02 LTS, No GPU)

(Arunava Chakraborty) #1

Hi, I am trying to compile the pytorch library from source and I can’t get it to work.
I tried following the CONTRIBUTING.md.

And I got this

/usr/bin/ld: cannot find -lshm
/usr/bin/ld: cannot find -ltorch_python
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-g++' failed with exit status 1

I am doing this.

cd build/
cmake ..
make install
python3 ../setup.py develop

I am refraining from posting the entire log here, as I don’t want to bombard here. I can if needed.

How do I get it to work?

(Arunava Chakraborty) #2


/usr/bin/ld: cannot find -lshm
/usr/bin/ld: cannot find -ltorch_python
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-g++' failed with exit status 1

Please let me know what is the possible reason for this?

(Arunava Chakraborty) #3

Okay, somehow I did it from fresh and it looks like it works!!

I ran:

git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
sudo python3 setup.py develop

and it ended with the following

x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.6/torch/csrc/dl.o -o build/lib.linux-x86_64-3.6/torch/_dl.cpython-36m-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-3.6/torch/_C.cpython-36m-x86_64-linux-gnu.so -> torch
copying build/lib.linux-x86_64-3.6/torch/_dl.cpython-36m-x86_64-linux-gnu.so -> torch
Creating /usr/local/lib/python3.6/dist-packages/torch.egg-link (link to .)
Adding torch 1.1.0a0+90b8552 to easy-install.pth file
Installing convert-caffe2-to-onnx script to /usr/local/bin
Installing convert-onnx-to-caffe2 script to /usr/local/bin

Installed /home/arunava/Projects/Cloned/pytorch
Processing dependencies for torch==1.1.0a0+90b8552
Finished processing dependencies for torch==1.1.0a0+90b8552

when I run

python3 test/test_torch.py

It gives me

Traceback (most recent call last):
  File "test/test_torch.py", line 8, in <module>
    import torch
ModuleNotFoundError: No module named 'torch'

I think this is simple linking issue or something like that, some inputs will be invaluable!

(Martin Huber) #4

hey there,

I cant tell you the exact reason for you problem, but it is best practice to build pytorch in a clean anaconda environment. Here is how. Please report back if it helps

(Arunava Chakraborty) #5

I will definately give this a try. Just let me know one thing.

conda install -c pytorch magma-cuda92

This is only required if I have a gpu right?

(Martin Huber) #6

correct, leave this out if you dont

(Arunava Chakraborty) #7

Thanks a lot for getting back. I will try this out and share the outcome here :slight_smile:

(Arunava Chakraborty) #8

Hey there, I have just started the build in a fresh conda env and it is still undergoing.
Meanwhile, I tried to install back torch==1.0.1 using pip3 (in my local)

And this is what I get

Requirement already satisfied: torch in /home/arunava/Projects/Cloned/pytorch (1.1.0a0+90b8552)

WOOAAAH!! So I did install it!

Excited as f@$k I then tried to import it in by running it in ipython3

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-eb42ca6e4af3> in <module>
----> 1 import torch

ModuleNotFoundError: No module named 'torch'

This looked like a common thing and stackoverflow to the rescue
I ran sudo ipython

>>> import torch
/usr/bin/python3: symbol lookup error: /home/arunava/Projects/Cloned/pytorch/torch/lib/libtorch_python.so: undefined symbol: PyThread_tss_alloc

and ipython3 exits!!

I am not sure why this happens.

The build is now running in conda env and I am very hopeful that it will work! I will update. Meanwhile let me know if you know some fixes of the above issue, although won’t be required if the build in the conda env works.

(Arunava Chakraborty) #9

ITTTTT WWWOOOOOORRRRKKKKKKKEEEEEEEDDDDD!!! :heart_eyes::heart_eyes::heart_eyes::heart_eyes::heart_eyes::heart_eyes::heart_eyes::heart_eyes::money_mouth_face::money_mouth_face::money_mouth_face::star_struck::star_struck::star_struck::star_struck::star_struck::star_struck::confetti_ball::confetti_ball::confetti_ball::confetti_ball::confetti_ball::confetti_ball::confetti_ball::confetti_ball::confetti_ball::boom::boom::boom::dizzy::dizzy::dizzy::dizzy::tada::tada::tada::tada::tada::tada::tada::tada::tada::sparkles::sparkles::sparkles::tada::tada::tada::tada:

THANKS A TOOON!! @mhubii :slight_smile: :slight_smile: :slight_smile:
So, running it in a conda env. That’s the cutie solution :slight_smile:

P.S. I went a bit crazy after I got it running, trying to do this from past 3-4 days :slight_smile: