I’m designing an Optimizer that needs to compute the 2nd derivative of parameters. but when I print the p.grad.grad it I got
for group in self.param_groups:
for p in group['params']:
grads2 = p.grad.grad
Has there any method to solve this?
You can set the
create_graph=True flag while running the backward pass to be able to backward through the backward pass.
But you won’t get a
In general, when doing higher order derivatives, I would suggest that you use
autograd.grad() that returns the gradients directly. That will avoid any confusion about gradient accumulation coming from different backward passes.