Two Loss Functions with weight factor

Good evening,

I’m using at the moment for my approach two loss functions and weight them with a factor:

loss_1= loss_1_function(output, args)
loss_2= loss_2_function(output, args)

overall_loss_tensorboard = args.alpha_loss*loss_1.item() + (1-args.alpha_loss)*loss_2.item()
(args.alpha_loss*loss_1+ (1-args.alpha_loss)*loss_2).backward()

Is this the right way, how one would weight two losses and combine them?

Thanks and best regards

Hi Jonas, I am also in the process of tuning multiple loss functions. Your approach seems somehow manual tuning. I suggest you to find a coarse optimization for alpha. After, I suggest you to start from this article and try some automatic tuning method.


Hi Stefano,

Yep that’s right and the worst thing is that this alpha factor is just one of 5 parameters… So thanks a lot for the link! I also wanted to look into the field of Bayesian Hyperparameter Optimization.

However just to be sure, the approach how I factor the two losses is correct or?

Thanks and best regrads

The way you are weighing the two losses is correct.

1 Like