hello, everyone! Im trying to deploy sample text classification model on android device but encountered an issue that unable to figure out.
I followed article “Classifying Names with a Character-Level RNN” where we train model to classify words based on language
https://pytorch.org/tutorials/intermediate/char_rnn_classification_tutorial
(output is a <1 x n_categories> Tensor, where every item is the likelihood of that category)
Model was successfully trained and i can run predictions locally on my machine with correct results:
python predict.py Nasir
(-0.73) Arabic
(-2.03) French
(-2.32) Czech
To use model on mobile device, it needs to be converted to PTL format using this code:
import torch
from torch.utils.mobile_optimizer import optimize_for_mobile
model = torch.load(‘names-classification.pt’)
model.eval()
scripted_module = torch.jit.script(model)
optimized_scripted_module = optimize_for_mobile(scripted_module)
optimized_scripted_module._save_for_lite_interpreter(“names-classification-lite.ptl”)
As a results I have lite ptl model file, that Im trying to use it on device. But at this point im not confident that model was converted properly - was not able to find a way to run predictions on local machine with PTL model.
Next step is android code:
mModule = LiteModuleLoader.load(liteModelPath)
val bytes = userInputtedName.toByteArray(Charset.forName("UTF-8"))
val shape = longArrayOf(1, bytes.size.toLong())
val inputTensor = Tensor.fromBlobUnsigned(bytes, shape)
val outputTensor1 = mModule!!.forward(IValue.from(inputTensor))
It loads the model without errors, but then it crashes on last line where I call “forward”
com.facebook.jni.CppException: Expected Tensor but got None
Looks like the error is coming from here:
But at this point Im stuck and cant quite tell:
- what does this error mean?
- did i configure input sensor wrong?
- is lite version of model is broken?
- is there a mismatch between versions of pytorch I used to train model and mobile lite library?
More details:
Example of inputTensor (toString): Tensor([1, 10], dtype=torch.uint8)
pytorch_android_lite:1.13.1
torch Version: 2.3.1
torchtext Version: 0.18.0
python Version: 3.11.9