How can I use different losses to update different branches respectively, and sum grad together to update master(main) branch

autograd
15

The first is elegant and correct. ‘loss = loss1+loss2’ is ok.
The second way may consume more memory ‘loss1.backward(retain_graph=True)’ makes the whole branch(including the main_block) stores the intermediate variables