If I want to uses
Params = Params + inverse(J'*J + mu * I) * g instead of
Params = Params + mu * g,
J is a Jacobian matrix and
J' is transpose of
J(i,j) = d loss(i) / d params(j),
g is a gradient vector,
g = J' * loss
I is identity matrix,
mu is a scaler.
How can I compute jacobian matrix and
inverse(J'*J + mu * I) , and finally update parameters with the above formulation?