relating to the question Custom loss with trainable parameters, I have two questions:
- Range for A and B?
Is there a way to constraint A and B to positive values? If yes, is it possible to ensure their values range between 0 and 1?
- Initialization for A and B
What would be the best way to initialize these parameters?
I don’t think you will get anything out of it.
Suppose that you have two loss which are adverse to each other in a way.
Then your setup may stuck at a = 0, b =1, then you’re only minimizing your second loss.
But if you want, you can set up it as
f = #parameter
a = sigmoid(f)
b = 1-a
This way, weights are positive and will add up to 1.
If you don’t force them to have at least positive sum, they both will converge to zero, and you’re minimizing nothing.
either way you should see these weights as
hyper-parameter and grid sreach combine with cross validation is common way for tuning them.