I’m trying to deploy a PyTorch model (currently implemented in 0.4, but simple to convert to 1.0) on AWS Lambda [1]. What’s the best way to accomplish this? From my searches, I’ve found a few options.
Deploy with C++ (for high speed)
It is possible to include a PyTorch model in C++ and compile it to a binary, as demonstrated in this tutorial.
Deploy via ONNX to Caffe2 (obsolete)
Previously, there was the option to export the model into the ONNX format and execute it via Caffe2. However, Caffe2 is now merging with PyTorch, which makes this option obsolete?
Compile things from source to save space
Another option I saw was to compile CPU-only PyTorch from source, to reduce the dependency size. However, is this now obsolete, given the pre-built Pytorch CPU pre-built binaries are only 90 MB?
Other options?
What other options are there for deploying a PyTorch model to AWS lambda? I noticed Glow was recently released, but I don’t understand the state of the project, nor it’s intended use.
[1] For those who aren’t aware. AWS Lambda allows you to only charge for a server when you’re using it. This is in contrast to an EC2 instance, which is constantly running. However, as a trade-off you’re limited by the deployed package size (250 MB). Thus, it’s impractical to use PyTorch with all of it’s dependencies.