# How to modify the gradient manually?

In particular I need to modify it by multiplying it for another function.

`````` loss.backward()
# Here i need to access the gradients and modified it.
optimizer.step()``````
1 Like
1 Like

Thanks, but the response is not conclusive. I need to know how to implement somenthing like this in tensorflow.

``````grads_and_vars = opt.compute_gradients(loss, parameter_list)
``````

If i’ve understand it well, torch.backward() store the grandient in parameter.grad, so i’ve to access every parameter iterating long the module parameters list.

``````loss.backward()
for p in model.parameters():
p.grad *= C  # or whatever other operation
optimizer.step()
``````
13 Likes

Thanks.

And if i’ve two loss function and different operations to do?

For example, i’ve f_1 and f_2, and I want the learning signal (or the gradient to apply) to be D(f_1) + D(f_2) * C, where D(f_1) stand for derivative.

It should be:

loss2.backward()
for p in model.parameters():
loss1.backward()
optimizer.step()

This works because i compute the gradients in f_2 before than f_1, otherwise grad accumulate gradients and i can’t acces only f_2 part. There’s a way to do it regardless of order? So is there a way to access the gradient before it is accumulated in p.grad?

``````loss2.backward(retain_graph=True)
for p in model.parameters():
optimizer.step()
loss1.backward()
optimizer.step()
``````

so you can avoid accumulate gradients on parameters.

1 Like

I see that this allows us to update the weights twice.
But what I want to weight the gradient from loss2 with C2 and from loss1 with C1 and make a single update?

3 Likes

``````loss1.backward(torch.ones_like(loss1)*C1)