Is there a way to get the gradient of all the “parent” nodes of a node during the execution of backprop (when viewing the graph going backward, e.g. the output node is the root and the input nodes are the leaves)?
For example, in the graph of this expression,
c = a * b out = a + c
a has two paths from
out -> a, one directly to
out and one through
c (shown below).
a ------- out \ / \ / \ / \ / b----c
I’d like to manipulate the gradient that gets computed for
a by running an arbitrary operation on the gradients accumulated from the parents of
a, in this case
out, probably by using a prehook on the nodes in the computation graph.
While I can get the parent grad_fns of the
grad_fn of a node, I’m having trouble figuring out how to get the associated gradient with that node. I was originally hoping I could do
parent_grad_fn.variable.grad, but it looks like these parent grad functions don’t have a
I’ve also attached a minimal example in colab here showing how I’ve been able to access the parent grad functions but not the actual grad value associated with them.
Thanks for the help!