what I do is
totalLoss = g_cost + G_mseCost
and then I take the gradient of the total loss
for name, f in G.named_parameters():
before I apply update
in this case the gradient I’m having is the total gradient. is there a way to get the gradient of one out of them by not doing two backward and if it is the only way how can I do the two backward without inferring again?
@chenyuntc so you are suggesting to use two backwards ?
To do two backwards passes you would need to pass
retain_graph=True to the first one.
I can’t see any way to only do one backwards pass.
In this case I should do
and it will take the two gradients in the update or I should add them first into new loss?
Each time you do
something.backward() the gradients are added to the gradients stored in
param.grad for each parameter.
So you can do
optimizer.step() # uses combined gradients from both losses