iArunava
(Arunava Chakraborty)
April 4, 2019, 2:55am
1
Hi, I am trying to compile the pytorch library from source and I can’t get it to work.
I tried following the CONTRIBUTING.md
.
And I got this
.
.
.
/usr/bin/ld: cannot find -lshm
/usr/bin/ld: cannot find -ltorch_python
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-g++' failed with exit status 1
I am doing this.
cd build/
cmake ..
make install
python3 ../setup.py develop
I am refraining from posting the entire log here, as I don’t want to bombard here. I can if needed.
How do I get it to work?
iArunava
(Arunava Chakraborty)
April 4, 2019, 5:19am
2
Hey,
/usr/bin/ld: cannot find -lshm
/usr/bin/ld: cannot find -ltorch_python
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-g++' failed with exit status 1
Please let me know what is the possible reason for this?
iArunava
(Arunava Chakraborty)
April 5, 2019, 6:29am
3
Okay, somehow I did it from fresh and it looks like it works!!
I ran:
git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
sudo python3 setup.py develop
and it ended with the following
.
.
.
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.6/torch/csrc/dl.o -o build/lib.linux-x86_64-3.6/torch/_dl.cpython-36m-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-3.6/torch/_C.cpython-36m-x86_64-linux-gnu.so -> torch
copying build/lib.linux-x86_64-3.6/torch/_dl.cpython-36m-x86_64-linux-gnu.so -> torch
Creating /usr/local/lib/python3.6/dist-packages/torch.egg-link (link to .)
Adding torch 1.1.0a0+90b8552 to easy-install.pth file
Installing convert-caffe2-to-onnx script to /usr/local/bin
Installing convert-onnx-to-caffe2 script to /usr/local/bin
Installed /home/arunava/Projects/Cloned/pytorch
Processing dependencies for torch==1.1.0a0+90b8552
Finished processing dependencies for torch==1.1.0a0+90b8552
But
when I run
python3 test/test_torch.py
It gives me
Traceback (most recent call last):
File "test/test_torch.py", line 8, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
I think this is simple linking issue or something like that, some inputs will be invaluable!
mhubii
(Martin Huber)
April 5, 2019, 7:55am
4
hey there,
I cant tell you the exact reason for you problem, but it is best practice to build pytorch in a clean anaconda environment. Here is how . Please report back if it helps
1 Like
iArunava
(Arunava Chakraborty)
April 5, 2019, 8:36am
5
I will definately give this a try. Just let me know one thing.
conda install -c pytorch magma-cuda92
This is only required if I have a gpu right?
mhubii
(Martin Huber)
April 5, 2019, 8:38am
6
correct, leave this out if you dont
iArunava
(Arunava Chakraborty)
April 5, 2019, 8:39am
7
Thanks a lot for getting back. I will try this out and share the outcome here
1 Like
iArunava
(Arunava Chakraborty)
April 6, 2019, 2:38am
8
Hey there, I have just started the build in a fresh conda
env and it is still undergoing.
Meanwhile, I tried to install back torch==1.0.1
using pip3
(in my local)
And this is what I get
Requirement already satisfied: torch in /home/arunava/Projects/Cloned/pytorch (1.1.0a0+90b8552)
WOOAAAH!! So I did install it!
Excited as f@$k I then tried to import it in by running it in ipython3
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-1-eb42ca6e4af3> in <module>
----> 1 import torch
ModuleNotFoundError: No module named 'torch'
This looked like a common thing and stackoverflow to the rescue
I ran sudo ipython
>>> import torch
/usr/bin/python3: symbol lookup error: /home/arunava/Projects/Cloned/pytorch/torch/lib/libtorch_python.so: undefined symbol: PyThread_tss_alloc
and ipython3
exits!!
I am not sure why this happens.
The build is now running in conda env and I am very hopeful that it will work! I will update. Meanwhile let me know if you know some fixes of the above issue, although won’t be required if the build in the conda env works.
iArunava
(Arunava Chakraborty)
April 6, 2019, 4:12am
9
2 Likes