Currently, I am pursuing a regression problem where I am attempting to estimate the time derivative of chemical species undergoing reaction and I am having a issue with the scales of my output. Since this is a detonation reaction, my outputs can range from essentially 0 for most cases to very large for others (during a detonation).
I think I am having problems with MSE as the loss function for this problem, since what I am really after is a low percentage error on each derivative estimation. If the output is meant to be 0.01 and the network predicts 10, I would like to punish this much more heavily than if the output is meant to be 1000 and the network predicts 1010. As of right now, I suspect that these large derivatives are dominating due to large contributions to loss.
Is there a loss function that implements a relative error loss based on training data? If I am correct in thinking that there is not, what is the recommended way to define a custom loss function?