How to avoid sum from ‘autograd.grad’ output in Physics Informed Neural Network?

[AlphaBetaGamma96]

@Isabella_Osei

“RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn”

Using the torch.func namespace requires you to re-write how you compute the gradients entirely, you can’t mix and match between torch.autograd and torch.func, especially when using torch.func.vmap (at least to my knowledge)

I have some previous examples on the forums of how to compute gradients with a ‘functional’ approach here:

Efficient computation of Hessian with respect to network weights using autograd.grad and symmetry of Hessian matrix

The documentation of the hessian function for functorch can be found here: Jacobians, Hessians, hvp, vhp, and more: composing functorch transforms — functorch nightly documentation I would advise using torch.func instead (if you’re using a more recent version of PyTorch), but something similar is possible in functorch. In torch.func an example of computing the Hessian with respect to parameters would be something like, import torch from torch import nn from torch.func import functional_call, v…

Autograd row-wise of a tensor using PyTorch autograd and without for loop

Hi @Arka_Roy, So, you’re mixing torch.autograd with torch.func operations, which leads to a zero gradient. You need a complete torch.func approach in order for torch.func to return the correct gradient. An example can be found below, import torch from torch.func import jacrev, vmap device = 'cuda' if torch.cuda.is_available() else 'cpu' def func(x): y1=x**2 y2=x**3 return torch.stack([y1,y2],dim=-1) x = torch.linspace(0,1,10, dtype=torch.float32, device=device) #don't need requires…