Perform EMA on loss

I calculate CE Loss in every batch but wish to use EMA on it for more stable training. How can I do so? All implementations do EMA over the model and not on the loss