Higher-order derivatives of torch::nn::Module

Hello! I was trying to translate my PyTorch code into LibTorch.
Using the following PyTorch code I was computing second-order derivatives of nn.modules.

import torch
from torch.autograd import Variable

def derivative(u,x,order = 1):
    ones  = torch.ones_like(u)
    deriv = torch.autograd.grad(u, x, create_graph=True, grad_outputs=ones)[0]
    for i in range(1,order):
        ones  = torch.ones_like(deriv)
        deriv = torch.autograd.grad(deriv, x, create_graph=True, grad_outputs=ones)[0]
    return deriv

Now I am trying to write the same function in LibTorch:

torch::Tensor derivative(torch::Tensor u, torch::Tensor x, int order = 1)
{
  torch::Tensor ones = torch::ones_like(u);
  torch::Tensor deriv = torch::autograd::grad({u}, {x}, /*grad_outputs=*/{ones}, /*create_graph=*/true, /*allow_unused=*/false)[0].requires_grad_(true);
  for (int i = 0;i < order; i++)
  {
    torch::Tensor ones = torch::ones_like(deriv);
    deriv = torch::autograd::grad({deriv}, {x}, /*grad_outputs=*/{ones}, /*create_graph=*/true, /*allow_unused=*/false)[0];
  }
  return deriv;
}

If I omit, the requires_grad_(true) I get an error that the Tensor deriv does not require a gradient, although this was the case in PyTorch… Thus I added requires_grad_(true).
Now I get the error:

One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.

I don’t want to set allow_unused=true in my C++ code, since then I don’t think that my second derivatives will be correct anymore. In Python it all worked with allow_unused=False.

How can I fix my errors and compute the second derivative of a torch::nn::Module in LibTorch correctly?

This sounds very fishy and I would double check.

Yes, you are right. It was an error on my side, I was computing the third derivative instead of the second derivative… Now everything works properly

Sounds like an error I’d put in my code, too. :slight_smile: I’m glad you solved it.