TypeError: The outputs of the user-provided function given to hessian must be either a Tensor or a tuple of Tensors but the given outputs of the user-provided function has type <class 'Functions.Determinant'>

Hi All,

I’m trying to calculate the hessian of a custom autograd Function via the use of torch.autograd.functional.hessian(Determinant. A). However, I’m getting the following error!

import torch
from Functions import Determinant

A = torch.randn(2,2, requires_grad=True)

psi_t = torch.det(A)
psi_c = Determinant.apply(A)

print(psi_t, psi_c)

grad1_t = torch.autograd.grad(psi_t, A)
grad1_c = torch.autograd.grad(psi_c, A)

print(grad1_t, grad1_c)

grad2_t = torch.autograd.functional.hessian(torch.det, A) #works
print(grad2_t)
grad2_c = torch.autograd.functional.hessian(Determinant, A) #fails

where Determinant is a torch.autograd.Function with its own Backward and DoubleBackward method. How exactly can I calcuate the hessian via torch.autograd.functional.hessian with a custom torch.autograd.Function?

The full stacktrace is,

Traceback (most recent call last):
  File "test_custom_det.py", line 18, in <module>
    grad2_c = torch.autograd.functional.hessian(Determinant, A)
  File "~/anaconda3/lib/python3.8/site-packages/torch/autograd/functional.py", line 701, in hessian
    res = jacobian(jac_func, inputs, create_graph=create_graph, strict=strict, vectorize=vectorize)
  File "~/anaconda3/lib/python3.8/site-packages/torch/autograd/functional.py", line 482, in jacobian
    outputs = func(*inputs)
  File "~/anaconda3/lib/python3.8/site-packages/torch/autograd/functional.py", line 697, in jac_func
    jac = jacobian(ensure_single_output_function, inp, create_graph=True)
  File "~/anaconda3/lib/python3.8/site-packages/torch/autograd/functional.py", line 482, in jacobian
    outputs = func(*inputs)
  File "~/anaconda3/lib/python3.8/site-packages/torch/autograd/functional.py", line 685, in ensure_single_output_function
    is_out_tuple, t_out = _as_tuple(out, "outputs of the user-provided function", "hessian")
  File "~/anaconda3/lib/python3.8/site-packages/torch/autograd/functional.py", line 21, in _as_tuple
    raise TypeError("The {} given to {} must be either a Tensor or a tuple of Tensors but the"
TypeError: The outputs of the user-provided function given to hessian must be either a Tensor or a tuple of Tensors but the given outputs of the user-provided function has type <class 'Functions.Determinant'>.

Thank you! :slight_smile:

Solution: I need to wrap my custom function within a nn.Module or a function to make it callable, for example,

def custom_det(A):
  return Determinant.apply(A)

then,

hessian = torch.autograd.functional.hessian(custom_det, A)