I’d like to compile a pytorch model to a standalone binary so that it would be extremly easy to deploy. I’ve seen this video Bin Bao at Modular Hack Weekend: From PyTorch Models to Standalone Binaries that is exactly on this topic but there is no code or tutorial on how to do it and I don’t find any documentation about that elsewhere.
How can I compile a pytorch model to a standalone binary ?
I haven’t watched the presentation in detail but I assume the author introduced AOTInductor described here:
AOTInductor is a specialized version of TorchInductor, designed to process exported PyTorch models, optimize them, and produce shared libraries as well as other relevant artifacts. These compiled artifacts are specifically crafted for deployment in non-Python environments, which are frequently employed for inference deployments on the server side.
In this tutorial, you will gain insight into the process of taking a PyTorch model, exporting it, compiling it into an artifact, and conducting model predictions using C++.