I’ve followed the tutorials and successfully created an example program. While building the program, I encountered several common bugs like missing MKL-DNN libraries and fixed them by building it from the source. Finally, everything works perfectly on my computer
However, when I want to migrate it to a clean computer, the program crashed. The reported error was:
Dyld Error Message:
Library not loaded: @rpath/libc10.dylib
I found it’s because of missing dynamic libraries in libtorch, but I cannot assume everyone has it.
I think the possible solutions are:
compile the program by static linking.
include the .dll/.dylib with the program
I am not familiar with C++. Are the proposed solutions correct? If so, how to achieve that, or what flags and path should I set in the CMAKE? I want my program being portable and no need to install extra dependencies on new computers.
[Disclaimer: I have neither built torch nor pytorch.]
Yes, these are the two methods you can use to make your program
portable. Note, you can use a mixture of the two methods – statically
link to some libraries, and provide dynamic libraries for other libraries.
I don’t know. I’ve never used CMAKE.
Please note that for portability, you will have to build your program
separately for each operating system you want to run on, and
possibly for different hardware platforms and different versions of
the same operating system.
You would normally do this by building on each of the platforms you
want to run on. (You can also “cross-compile” and “cross-build,” but
that is an art in itself.)
If you want your program to be casually portable, you can build on
your target systems, and hope for the best. If you want it to be
commercial-grade portable, you must test, test, test on various
combinations of operating system, hardware, and version and
configuration of operating system. For commercial-grade portability
you will probably also want to distribute your program as installers
and package-manager packages appropriate for your target systems.
My current plan is to write an MNIST classifier with GUI as a prototype and try to make it become commercial-grade portable, if possible. I think it would be the best template to try the things you mentioned.
Is this practical by merely using libtorch? Or should I try ONNX or something else? Is pytorch/libtorch suitable for this scenario or should I try other frameworks?
Actually, I have no idea how the industrials deploy deep learning models. But because I can write my own CMakeLists.txt, it’s very simple and convenient to compile with other C++ frameworks (e.g., JUCE, QT, …). And this gives me a strong motivation to try libtorch.
Hello Wen-Yi,
In addition to the wonderful advice provided by @KFrank, I’ll add the following:
You will find that people deploy ML models in a variety of ways, but an approach preferred by many is to train models in Python and run inference in C++. Libtorch has a really nice C++ API and you can write code to construct and train models in C++, as well as to run inference. In my environment, I use C++ for training as well as for inference.
If you do take the C++ route, I would encourage you to compile Libtorch from source and ensure that your LD_LIBRARY_PATH/DYLD_LIBRARY_PATH is set to point to the correct location of your Libtorch libraries. Some folks prefer statically linking, but I always link dynamically, taking care to ensure that my LD_LIBRARY_PATH is correctly set. One thing that CMake will need is to know how to find your Libtorch libraries, and the CMakelists.txt file in the tutorial (https://pytorch.org/tutorials/advanced/cpp_frontend.html) shows how to accomplish that.
Once you get the base build going, you can move onto building with CUDA/ROCM if you want acceleration.
The C++ tutorial is nicely written and should provide you all you need to get going. In the event you really do embrace the C++ route, I would suggest that you took a look at Conan (http://conan.io), which is a popular technique to build, package, and deploy C++ libraries. You can look at my github repo (github.com/forwardmeasure) for the Conan recipe for Libtorch.
