how to get gradients that still have requires_grad True

[yash11 (yash kumar)]

Thanks for your reply.

I actually want gradient of wp only wrt phi, so this worked for me,

w = Parameter(T.tensor([2.2]))
phi = Parameter(T.tensor([1.5]))
wp = w*phi
grd = T.autograd.grad(wp, phi, create_graph=True)[0]
print(grd)
grd.backward()
w.grad
print(w.grad)

output:

tensor([2.2000], grad_fn=<MulBackward0>)
tensor([1.])

Using modified last method

w = Parameter(T.tensor([2.2]))
phi = Parameter(T.tensor([1.5]))
wp = w*phi
wp.backward(create_graph=True)
grd = phi.grad
print(grd)
grd.backward()
w.grad
print(w.grad)

output:

tensor([2.2000], grad_fn=<CopyBackwards>)
tensor([2.5000], grad_fn=<CopyBackwards>)

I don’t know what is going on with last method.
Also I found a Quote, which suggest against using .grad in such cases.

@albanD

If you have a single input and single output, you want to do the following:
(note that using .backward() for higher order derivatives is discouraged because the .grad field becomes hard to reason about).

first_derivative = autograd.grad(loss, x, create_graph=True)[0]
# We now have dloss/dx
second_derivative = autograd.grad(first_derivative, x)[0]
# This computes d/dx(dloss/dx) = d2loss/dx2