Hi! I am new in PyTorch. I want to update the weight by a old grad:
Here is a pseudocode example:
output = model(input)
loss1 = criterion1(output, target)
loss2 = criterion2(output, target)
grad1 = loss1.compute_grad()
grad2 = loss2.compute_grad()
parameters.update(grad1)
... # other operation will be done here.
parameters.update(grad2)
The key point is that the grad2 is calculated w.r.t the old weight, before the parameters.update(grad1)
.
I have no ideas to do this in an efficient way. Can create_graph=True
or other method in Pytorch can help me? An easy example is better. Thank you!