Register_hook and find the maximum gradient of non leaf variable

[Blade6570 (Soumya Ranjan Tripathy)]

I have put the register_hook() for a non_leaf variable and printing the gradient. How to access each element in the gradient tensor.

The syntax I am using is as follows:

y.register_hook(print)

[Giuseppe_Castiglione]

You’ll find the gradient by looking at y.grad. Usually this isn’t saved though, so take a look here:

Why cant I see .grad of an intermediate variable?

Hi! I am loving this framework… Since Im a noob, I am probably not getting something, but I am wondering why I cant get the gradient of an intermediate variable with .grad? Here is an example of what I mean: xx = Variable(torch.randn(1,1), requires_grad = True) yy = 3*xx zz = yy**2 zz.backward() xx.grad # This is ok yy.grad # This gives 0! zz.grad # This should give 1! So I get the correct result for xx.grad, but why does yy.grad show 0, as does zz.grad? How can I get the yy.grad value in t…

If you want to specifically use a hook, then you can register a closure as a hook and that will save the gradient, As was done here:

Why cant I see .grad of an intermediate variable?

You can think of a function that also keeps some additional variables from the outer scope. For example in here hook is a closure that remembers a name given to the outer function: grads = {} def save_grad(name): def hook(grad): grads[name] = grad return hook x = Variable(torch.randn(1,1), requires_grad=True) y = 3*x z = y**2 # In here, save_grad('y') returns a hook (a function) that keeps 'y' as name y.register_hook(save_grad('y')) z.register_hook(save_grad('z')) z.backward()…

[Blade6570 (Soumya Ranjan Tripathy)]

Hi, Thanks for the response. I got it now.