I’m using at the moment for my approach two loss functions and weight them with a factor:
loss_1= loss_1_function(output, args)
loss_2= loss_2_function(output, args)
overall_loss_tensorboard = args.alpha_loss*loss_1.item() + (1-args.alpha_loss)*loss_2.item()
Is this the right way, how one would weight two losses and combine them?
Thanks and best regards
Hi Jonas, I am also in the process of tuning multiple loss functions. Your approach seems somehow manual tuning. I suggest you to find a coarse optimization for alpha. After, I suggest you to start from this article https://towardsdatascience.com/self-paced-multitask-learning-76c26e9532d0 and try some automatic tuning method.
Yep that’s right and the worst thing is that this alpha factor is just one of 5 parameters… So thanks a lot for the link! I also wanted to look into the field of Bayesian Hyperparameter Optimization.
However just to be sure, the approach how I factor the two losses is correct or?
Thanks and best regrads
The way you are weighing the two losses is correct.