Build shared lib fail at linking stage

windows
1

Hello there,

I am trying to build the lite version in Release on Windows. The goal is to get a smaller shared lib that focus on the inference part of Pytorch.
After trying multiple times, both following README instructions from the repository and using the builder available on Git, I’m reaching a point where everything seems to build correctly but the lib generation fails at linking stage.
Does anyone know what’s happening here ?
Thanks for reading me !

I’m providing the logs after removing a few lines of unresolved external symbols for clarity :

[6837/7346] Linking CXX shared library bin\torch_cpu.dll
FAILED: bin/torch_cpu.dll lib/torch_cpu.lib 
C:\Windows\system32\cmd.exe /C "cd . && "C:\Program Files\CMake\bin\cmake.exe" -E vs_link_dll --intdir=caffe2\CMakeFiles\torch_cpu.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\mt.exe --manifests  -- C:\PROGRA~1\MICROS~4\2022\COMMUN~1\VC\Tools\MSVC\1437~1.328\bin\Hostx64\x64\link.exe /nologo @CMakeFiles\torch_cpu.rsp  /out:bin\torch_cpu.dll /implib:lib\torch_cpu.lib /pdb:bin\torch_cpu.pdb /dll /version:0.0 /machine:x64 /ignore:4049 /ignore:4217 /ignore:4099 /INCREMENTAL:NO  -WHOLEARCHIVE:C:/Users/juliena/Desktop/builder/windows/pytorch/build/lib/caffe2_protos.lib -WHOLEARCHIVE:C:/Users/juliena/Desktop/builder/windows/pytorch/build/lib/onnx.lib  && cd ."
LINK: command "C:\PROGRA~1\MICROS~4\2022\COMMUN~1\VC\Tools\MSVC\1437~1.328\bin\Hostx64\x64\link.exe /nologo @CMakeFiles\torch_cpu.rsp /out:bin\torch_cpu.dll /implib:lib\torch_cpu.lib /pdb:bin\torch_cpu.pdb /dll /version:0.0 /machine:x64 /ignore:4049 /ignore:4217 /ignore:4099 /INCREMENTAL:NO -WHOLEARCHIVE:C:/Users/juliena/Desktop/builder/windows/pytorch/build/lib/caffe2_protos.lib -WHOLEARCHIVE:C:/Users/juliena/Desktop/builder/windows/pytorch/build/lib/onnx.lib /MANIFEST:EMBED,ID=2" failed (exit code 1120) with the following output:
   Creating library lib\torch_cpu.lib and object lib\torch_cpu.exp
AdaptiveAvgPoolKernel.cpp.AVX2.cpp.obj : error LNK2001: unresolved external symbol "void __cdecl at::share_memory_(class at::TensorBase &)" (?share_memory_@at@@YAXAEAVTensorBase@1@@Z)
Activation.cpp.AVX2.cpp.obj : error LNK2001: unresolved external symbol "void __cdecl at::share_memory_(class at::TensorBase &)" (?share_memory_@at@@YAXAEAVTensorBase@1@@Z)
BinaryOpsKernel.cpp.AVX2.cpp.obj : error LNK2001: unresolved external symbol "void __cdecl at::share_memory_(class at::TensorBase &)" (?share_memory_@at@@YAXAEAVTensorBase@1@@Z)

... (a lot of unresolved external sympbols here)

sgd.cpp.obj : error LNK2001: unresolved external symbol "public: virtual void __cdecl torch::nn::Module::load(class torch::serialize::InputArchive &)" (?load@Module@nn@torch@@UEAAXAEAVInputArchive@serialize@3@@Z)
sgd.cpp.obj : error LNK2001: unresolved external symbol "public: virtual void __cdecl torch::nn::Module::pretty_print(class std::basic_ostream<char,struct std::char_traits<char> > &)const " (?pretty_print@Module@nn@torch@@UEBAXAEAV?$basic_ostream@DU?$char_traits@D@std@@@std@@@Z)
sgd.cpp.obj : error LNK2001: unresolved external symbol "public: virtual bool __cdecl torch::nn::Module::is_serializable(void)const " (?is_serializable@Module@nn@torch@@UEBA_NXZ)

sgd.cpp.obj : error LNK2001: unresolved external symbol "private: virtual void __cdecl torch::nn::Module::clone_(class torch::nn::Module &,class std::optional<struct c10::Device> const &)" (?clone_@Module@nn@torch@@EEAAXAEAV123@AEBV?$optional@UDevice@c10@@@std@@@Z)

bin\torch_cpu.dll : fatal error LNK1120: 80 unresolved externals
2

Is there anyone that can help me please ?
I can provide more logs if necessary… I really need to compile a lite version and I can’t make it using the provided scripts

Thank you again :pray:

1 Like
3

I have exact same error, were you able to solve this ?

4

I was indeed
A bit sad noone answered that one actually. The fact is, using the BUILD_LITE_INTERPRETER flag is working for mobile only and not for Windows. You simply can’t get inference only builds which is kind of sad really… Still working on trying to reduce the size of the output dll which grows quite fast when including a couple GPU architectures

5

Completely fine on my side. I haven’t tried rebuilding on windows yet but I was able to build pytorch on WSL Ubuntu 22.04. Looking at my last settings, I already have it :

export USE_CUDA=1 USE_CUDNN=1 USE_MKLDNN=0 USE_DISTRIBUTED=0 DEBUG=1  BUILD_SHARED_LIBS=ON
export USE_NINJA=0 USE_OPENMP=0 MAX_JOBS=3 USE_FLASH_ATTENTION=0  USE_ITT=0 USE_OBSERVERS=0 VERBOSE=1
export USE_MEM_EFF_ATTENTION=0 BUILD_TEST=0 USE_GLOO=0 USE_XNNPACK=0 USE_MIMALLOC=0 USE_NCCL=0
export USE_FBGEMM=0 USE_PYTORCH_QNNPACK=0 USE_VULKAN=0 BUILD_LITE_INTERPRETER=0 USE_KINETO=0

I think I’m going to give it a try again and will give here an update : Maybe, my build problem is diffrent to yours — but yeah my reply might take a while since im currently working on different problem. Thanks for the update.

6

Thank you for the update. Indeed you are not trying to build the lite interpreter so this is not the same issue. My advice would be to take a peak in .ci/pytorch and look at the different scripts to see what you might have missed in the environment setup and more importantly to start with default values of the flags as defined in CMakeLists.txt and try to change the flags one by one to see which one is causing your issue.

Good luck !