I am trying to get my head around if optimizing a combination two loss function with one with positive values and the other with negative values is possible. Basically, the problem is to minimize f = f1 + f2 where f1 has positive values and f2 has negative values. So minimizing f is equivalent to minimizing f1 (getting close to zero) and maximizing -f2 (which means minimizing f2 and getting larger negative values). Specifically, the flow of data in my classification problem is as below:
where xi is the feature for one input (network output). CE is crossentropy loss. My custom loss is a custom function of xi and some additional parameters that outputs negative values and total loss = CE + alpha * CustomLoss.
I’m eager to know your comments on this model.