The second order derivative of a function with respective to the input

autograd
1

Hi,

I have a question about the second order derivative of with respective to the input. Say,

class Test(nn.Module):
    def __init__(self):
        super().__init__()
        
        bo_b = False
        bo_last = False

        self.l1 = nn.Linear(1, 1, bias = bo_b).to(device)

        self.l2 = nn.Linear(4, 1, bias = bo_b).to(device)
        
    def forward(self, v):
        v = self.l1(v) 
        v = self.l2(v) 
        return v

fnn = Test()

xx = [[0], [0.5], [1]]
xx_torch = torch.tensor(xx, requires_grad = True, device = device, dtype=torch.float32)

ge_out = fnn.forward(xx_torch)
df_dx = torch.autograd.grad(ge_out, xx_torch, 
                                   grad_outputs = torch.ones( xx_torch.size()), create_graph = True)[0]
df2_dx2 = torch.autograd.grad(  df_dx, xx_torch )[0]

But it gives me the error message: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.

Can anyone help me with the problem? Thanks a lot!

2

Hi,

The problem is that your function is linear. So the first gradient is constant and the second order gradient is independent of the input.
This error message happens because of the independence (and thus, it is not used in the graph).

1 Like
3

Hi, Alban,

Thanks for your reply! I have an additional question. I see some post and they use df_dx.sum() instead of the df_dx in the torch.autograd.grad(), do you know why is that? Thanks!

4

It is equivalent to passing grad_outputs = torch.ones( xx_torch.size()) as you do.

1 Like