PyTorch on ppc64l2 cpu-only systems

I have installed PyTorch on P9 (ppc64le cpu-only). Since binaries are not available for this system, I installed it from scratch. Conda list shows:
torch 2.4.0a0+git7ddf57e dev_0
However, when I run an LLM using this, it runs extremely slow (110x slow). Can someone suggest what could be wrong and what measures I should take to rectify.