Hello all, I am in need of building Pytorch 1.6.0 from source offline.
Originally, this came up because some machines utilizing pytorch needed libtorch (the conda distribution of Pytorch 1.6.0 was already on the systems), there is a lengthy approval for our IT. Finally, libtorch was brought onboard, however it is 1.7.1.
When trying to forward propagate on a model in c++ which was serialized on Pytorch and loaded by libtorch my program simply errors out. After digging around online I am wondering if it is due to version mismatch. Therefore (since Pytorch 1.6.0 has already been approved by our IT), I will just build from source Pytorch/Libtorch. One issue, I have to do this offline.
Before I brought Pytorch in I:
git clone https://github.com/pytorch/pytorch.git
git checkout v1.6.0
git submodule update --init --recursive
Then I moved my distribution to the offline machine. Once on the machine:
set CMAKE_GENERATOR=Visual Studio 16 2019
set DISTUTILS_USE_SDK=1
for /f "usebackq tokens=*" %i in (`"%ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe" -version [15^,16^) -products * -latest -property installationPath`) do call "%i\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=%CMAKE_GENERATOR_TOOLSET_VERSION%
set CUDAHOSTCXX=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX64\x64\cl.exe
python setup.py build --cmake
Now, this will be as best as I can put it, I cannot copy paste mass log dumps due to the offline restriction but it appears to run as normal until:
Performing download step (git clone) for 'psimd'
Which, obviously fails. Then the procedure continues and eventually crashes, after checking the CmakeErrorLog.txt I am shown errors such as
cannot find include file 'pthread.h'
I am marking this as deployment since that seems to be the center of what I am describing here, however, as the reader can see the root of my issue stems from (what I think is) pre-compiled release Pytorch 1.6.0 not playing nicely with pre-compiled libtorch 1.7.1, if anyone has insight there I would much appreciate it.
Expected behavior:
- everything work normally, I can either use my existing pytorch/libtorch distributions fine no problem, OR I can build from source offline with no issues
My Environment:
- Windows 10
- Visual Studio 16 2019
- Python 3.7
- Everything that came with anaconda
Additional notes:
- torchvision is not on the system so no I cannot follow the exact minimum reproducible example exactly for deploying a model to c++ from python