Thanks for your reply @alexis-jacq .
I tried your code, but still get unexpected result on my machine (pytorch0.4):
Besides, after autograd.grad() is called, x.grad is None.
However, gradients must have been accumulated somewhere somehow during the calculation, but I don’t know how it is done (my second example in the original post is more tricky to find out how gradient is accumulated).
I know torch.autograd.backward() will accumulate gradients, should autograd.grad() also accumulate gradients? Apparently it does during the call, but the gradients won’t be save to variables.
Besides, my original goal of using autograd.grad is to calculate Hessian matrix (not a Hessian vector: pytorch/test/test_autograd.py at main · pytorch/pytorch · GitHub). Any suggestion on this is appreciated.