Hello everyone,
I am working on the Whisper ASR model inference and getting different results on two different CPU architectures. On an Intel CPU, I am able to get normal results. However, on an ARM CPU, the model is unable to predict normal tokens, resulting in empty text. After checking, I found that the GELU function output is different on both CPUs. The MSE error of GELU results in layer 1 is 0.0048.
I am using the pre-trained Whisper-tiny.en model directly from Hguuingface, and the Python and PyTorch versions are the same on both CPUs, with all relevant libraries installed.
The ARM CPU is Cortex-A53 on Xilinx KV260. The x86 CPU is Intel Core i7.
I would like to ask what could be the possible cause of this issue and how to resolve it? Thank you.