Hi there,
I am trying to speed up / vectorise the following code:
def function(input_):
# Compute prediction for u
for ii in range(input_.shape[0]):
x = input_[ii, :]
x.requires_grad = True
y = targets[ii]
u = model(x)
first_derivative = autograd.grad(u, x, retain_graph=True, create_graph=True)
hessian_mtrx = hessian(model, x, create_graph=False, strict=False)
I am confused about the interfaces as the hessian requires “sequences”(Automatic differentiation package - torch.autograd — PyTorch 2.1 documentation) but
x_tup = tuple(x.detach()) # with and without .detach()
hessian_mtrx = hessian(model, x_tup, create_graph=False, strict=False)
does not work and I get a Typeerror where the batch size is confused for the dimension of the input to the model:
TypeError: forward() takes 2 positional arguments but 33 were given
I am similarly confused about the correct and efficient usage of autograd.grad as
x = input_
x.requires_grad = True
u = model(x)
first_derivative = autograd.grad(u, x, retain_graph=True, create_graph=True)
gives the following error message:
RuntimeError: grad can be implicitly created only for scalar outputs
Any pointers or partial solutions are highly appreciated!
Thanks a lot!
model(x) takes an input of size (n,2) and returns a one-dimensional tensor of length n.