How to use backward for different layer of different loss function

autograd
1 
         | - loss1
         |

convs …fc3 - fc1 - fc2 - loss2
|
| - loss3

I have three loss functions, I use torch.autogra.backward([l1,l2,l3]).
Firstly, Is it right, meaning different loss update different layers and layers before the leaf node?
Secondly, if I want to sum this three loss, how can I do? L = l1 + l2 + l3 , L.backward()?
If I do in this way, L update from which node?

2

Yes, just simply do something like

loss_total = loss_1 + loss_2 + loss_3
loss.backward()

In this way, loss_total will update all the parameters.

More specifically, based on your figure, loss_1 will update convs to fc1, loss_2 will update convs to fc2 and loss_3 will update convs.

3

Thank you.
But if I use loss_total to update fc1 and layers before fc1 with fc2 and layers before unupdate for K steps , and then update fc2 and layer after fc1 with other layers unupdate. and repeat steps above. what can I do?