Pytorch models for Production

nlp
1

Deploying pytorch models to production ? I guess Pytorch is not production ready ? How to do it for [roduction purpose ?

2

It depends on the platform to which you’re aiming to deploy and some other constraints, for example if your use case can be fulfilled via REST or similar service and you don’t mind the python overhead you could potentially use PyTorch as it is on a server to handle web requests. However if you’re aiming for edge deployment or you want to squeeze as much raw performance as possible, your best bet right now is to use ONNX. You can export your PyTorch model in ONNX format and then use another framework (like caffe2, MXNet, CNTK, etc.) to actually run the model, these other frameworks do support edge and/or have specialized deployment extensions (e.g. the MXNet model server which can also serve ONNX models directly).

BTW PyTorch 1.0 is coming and will be production-ready, and I’m very excited about that! :smiley:

1 Like
3

thanks. when the pytorch 1.0 willl rereleased ?

4

As far as I know “sometime during the summer” is the only time frame announced.

5

Hi, just curious about the PyTorch 1.0. If I understand correct, ‘it’s ready for production’ means it can handle many requests simultaneously, say, 100 for a single server , right?

1 Like