How to weight my own multiple losses?

1

I have multiple losses, and i want to weight them seperately. More specifically, i want to set the weight of loss1 to 1, and the weight of loss2 to 0.5, is that correct?
loss1 = nn.L1Loss(B, A)
loss2 = nn.L1Loss(C, A)
loss2 = loss2*0.5
loss1.backward(retain_variables=True)
loss2.backward()
optimizer.step()

2 Likes
2

Hi,

Yes that would work.
By the way, a more efficient way to do it is:

loss1 = nn.L1Loss(B, A)
loss2 = nn.L1Loss(C, A)
final_loss = loss1*1 + loss2*0.5
# Don't forget the optimizer.zero_grad() somewhere before the backward
final_loss.backward()
optimizer.step()
3 Likes
3

Thanks for your reply:grinning:

4

do you maybe have any idea, that, the weight of loss can also be learnt by the net?
like:
loss1 = nn.L1loss(B,A)
loss2 = nn.L1loss(C,A)
final_loss = alpha * loss1 + beta * loss2
??
how to deal with the alpha and beta?

5

You can either have them as Parameter or add a network that outputs the two values.

1 Like