Computing the 2nd derivative of parameters

1

I’m designing an Optimizer that needs to compute the 2nd derivative of parameters. but when I print the p.grad.grad it I got None,

for group in self.param_groups:
    for p in group['params']: 
        grads2 = p.grad.grad

Has there any method to solve this?
Thank you.

2

Hi,

You can set the create_graph=True flag while running the backward pass to be able to backward through the backward pass.
But you won’t get a .grad.grad.
In general, when doing higher order derivatives, I would suggest that you use autograd.grad() that returns the gradients directly. That will avoid any confusion about gradient accumulation coming from different backward passes.