Slow inference on HuggingFace timm models

Hi folks
I am trying to do inference on Imagenete with retrained resnet50 models using an MPS background (Apple Mac M3 Pro). I am noticing some operations like batch_norm are taking bulk of operation. Adding screenshots for the same

Any insight will be appreciated to reduce this.