Can you apply transformations in Custom Loss functions

You could try to use a “soft” threshold as described here.