I am trying to speed up / vectorise the following code:
def function(input_): # Compute prediction for u for ii in range(input_.shape): x = input_[ii, :] x.requires_grad = True y = targets[ii] u = model(x) first_derivative = autograd.grad(u, x, retain_graph=True, create_graph=True) hessian_mtrx = hessian(model, x, create_graph=False, strict=False)
I am confused about the interfaces as the hessian requires “sequences”(https://pytorch.org/docs/stable/autograd.html) but
x_tup = tuple(x.detach()) # with and without .detach() hessian_mtrx = hessian(model, x_tup, create_graph=False, strict=False)
does not work and I get a Typeerror where the batch size is confused for the dimension of the input to the model:
TypeError: forward() takes 2 positional arguments but 33 were given
I am similarly confused about the correct and efficient usage of autograd.grad as
x = input_ x.requires_grad = True u = model(x) first_derivative = autograd.grad(u, x, retain_graph=True, create_graph=True)
gives the following error message:
RuntimeError: grad can be implicitly created only for scalar outputs
Any pointers or partial solutions are highly appreciated!
Thanks a lot!
model(x) takes an input of size (n,2) and returns a one-dimensional tensor of length n.