Static build that supports torchscript

C++
1

I made a custom build of Torch to link into an existing c++ application. This is the simplest build of torch that I could manage. To give you background, this is for a POS model that has already been trained but only needs to be implemented in the c++ program. When loading the model into the program I get the following error …
Unknown builtin op: aten::mul.
Could not find any similar ops to aten::mul. This op may not exist or may not be currently supported in TorchScript.
:
File “”, line 3

def mul(a : float, b : Tensor) → Tensor:
return b * a
~~~~~ <— HERE
def add(a : float, b : Tensor) → Tensor:
return b + a
‘mul’ is being compiled since it was called from ‘gelu_0_9’
File “”, line 3

def gelu_0_9(self: Tensor) → Tensor:
return torch.gelu(self, approximate=‘none’)
~~~~~~ <— HERE

As far as I can tell what this means I need to force the entire object file to be linked into my binary. That isn’t a problem. The problem is I don’t know which library has the aten::mul (and other)_operators in it. The compile I options I am using are as follows…

cmake ../ -DUSE_CUDA=OFF -DUSE_FBGEMM=OFF -DUSE_KINETO=OFF -DUSE_CUPTI_SO=OFF -DUSE_OPENMP=OFF -DUSE_NUMPY=OFF -DB UILD_SHARED_LIBS=OFF -DBUILD_CUSTOM_PROTOBUF=ON -DUSE_NNPACK=ON -DUSE_QNNPACK=OFF -DUSE_PYTORCH_QNNPACK=OFF -DUSE_ XNNPACK=OFF -DUSE_DISTRIBUTED=OFF -DUSE_MKLDNN=OFF -DBUILD_PYTHON=OFF -DBUILD_CAFFE2=OFF -DUSE_NCCL=OFF -DUSE_VALG RIND=OFF -DONNX_ML=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../../install

So everything is off. There maybe something I have turned off I need to support torchscript but for the life of me I can’t find figure out what it is. Any help with either the compile or which libraries are necessary to make this work would be very helpful. Thank you.

2

I figured it out. So for anyone who runs into this issue the first thing you are going to need to do is for the entire libtorch_cpu(or whatever gpu you are using) static library to be loaded into your binary. You can do this with ld using -Wl --whole-archive,. After you do that you will get a ton of linking errors.

Now when you use cmake --install . it quite sadly only puts some of the libraries that you need in the install path. In truth you are going to need nearly all of them. They will be in your cmake build directory/lib/ Here is the order I am linking them (order matters).
‘torch_cpu’,
‘torch’,
‘c10’,
‘protobuf’,
‘onnx’,
‘onnx_proto’,
‘cpuinfo’,
‘nnpack’,
‘pthreadpool’,
‘Caffe2_perfkernels_avx2’,
‘Caffe2_perfkernels_avx’,
‘sleef’,
‘ittnotify’

Important note on sleef. It is not in the /lib directory. It is however in /sleef/lib so be sure to grab it out of there. With that I have a static cpu build with TorchScript running in a c++ application. Enjoy.