Pytorch Mobile - Drastically different output for the same model between PC and Android

Hi everyone :slight_smile:

I just finished training a model with good results, and serialized it using torch.jit.trace(). However, the model performs very poorly when deployed to my Android phone.

While investigating the cause of the performance difference between devices, I ran both models (pc and android) using a zero-tensor as input. To my surprise, I found out the model outputs are totally different.

This would explain the poor results (since the mobile model essentially outputs nonsense values). But is this considered a bug or am I doing something wrong on my end?

for reference, here is the code I have written:

  1. After training the model (on my pc):
encoder = encoder.to(cpu).eval()
mock_encoder_input = torch.tensor(np.random.rand(1, 1, 138, 80), dtype=torch.float)
traced_encoder = torch.jit.trace(encoder, mock_encoder_input)
traced_encoder.save(colab_data + "/spectrograms/encoder.pt")
  1. Checking the model outputs on the pc:
encoder(torch.zeros(1, 1, 138, 80, dtype=test_spectrograms.dtype))

with the first 5 scalars in the output tensor being: -7.5927e-01, 1.1893e-02, -6.2612e-03, 5.3497e-03, -3.3745e-02

  1. Checking the model outputs on my Android phone:
IValue spectroTensor = IValue.from(Tensor.fromBlob(new float[(int)(dim0 * dim1 * dim2 * dim3)], tensorShape));
IValue[] encoderResults = encoder.forward(spectroTensor)

with the first 5 scalars in the output tensor being: 0.23662394, -0.41967326, 0.21712059, -0.312757, 0.29841155,

Can you verify the output of spectroTensor to see if it is really all zeros?

Do you mean the output of encoder.forward(spectroTensor)? Because it isn’t zero and shouldn’t be zero.

If you just mean the value of spectroTensor itself - yes, I have already verified it consists of zeros.

Well figured it out, and it had nothing to do with PyTorch

In short - Android will not update asset files when these are changed. You must manually clear the caches through the phone/emulator settings => apps menu (sigh…)

The app was running an older model (one of many trials), causing the discrepancies I saw.

1 Like