Does PyTorch support such type of inference?
This is currently not supported, but we are actively looking into supporting inference on GPU on mobile devices.
Stay tuned for future releases!
Any updates on this?
I recently read about ONNX runtime and it might be an alternative.
PyTorch could export models to ONNX format and run the model with ONNX runtime.
According to the onnxruntime, NNAPI is supported including CPU and GPU inference.
I didn’t try it but looks like it is the only Android gpu-enabled accelerator for now.