Mobile deployment best practice?

[seungjun]

Thanks for sharing your experience!
I typically work on image regression models that consume quite heavy computation.

While waiting for the official mobile GPU support, I guess your suggestion is the most practical way at the time being.

By the way, I found that GPU & NNAPI acceleration might be possible if we use onnx runtime.

Network inference on GPU under Android

I recently read about ONNX runtime and it might be an alternative. PyTorch could export models to ONNX format and run the model with ONNX runtime. https://pytorch.org/docs/stable/onnx.html According to the onnxruntime, NNAPI is supported including CPU and GPU inference. I didn’t try it but looks like it is the only Android gpu-enabled accelerator for now.

I will try both ways