| - loss1
convs …fc3 - fc1 - fc2 - loss2
| - loss3
I have three loss functions, I use torch.autogra.backward([l1,l2,l3]).
Firstly, Is it right, meaning different loss update different layers and layers before the leaf node?
Secondly, if I want to sum this three loss, how can I do? L = l1 + l2 + l3 , L.backward()?
If I do in this way, L update from which node?
Yes, just simply do something like
loss_total = loss_1 + loss_2 + loss_3
In this way,
loss_total will update all the parameters.
More specifically, based on your figure,
loss_1 will update convs to fc1,
loss_2 will update convs to fc2 and
loss_3 will update convs.
But if I use loss_total to update fc1 and layers before fc1 with fc2 and layers before unupdate for K steps , and then update fc2 and layer after fc1 with other layers unupdate. and repeat steps above. what can I do?