I have a C++ based project (that compiles in Windows, MacOS and Linux). I have a pre-trained model in pytorch and I am performing inference (only inference and no training) using this model in my C++ project. When I distribute a standalone installer for this C++ project, I also have to ship the pytorch dependencies (libtorch-cxx11-abi-shared-with-deps-1.3.0.zip) with my software. The current size of pytorch dependencies are 1.4GB. My application size is around 2MB.
Is there any possibility of creating a pytorch distributable library that only contains the minimal code required for inference (no training)? I feel that 1.4GB of dependency is too big and unnecessary if I only want to perform inference.
I would really appreciate if the members of this forum provide me with pointers and insight on how I can embed pytorch inference functionality in my C++ projects without the burden of 1.4GB of libtorch dependency.