# Unable to find minimum with optimizer

After being able to find the minimum value with scipy.optimize, I tried to implement the logic to find the optimal values with torch. When I tried to run the optimizer, it returned nan values.
This is the code that I used to find the minimum with scipy.optimize:

``````def get_pow_geometric(f, angle, power):
return np.sign(f(angle))*np.abs(f(angle)) ** power

def get_distance(latent_params, p, a, epsilon, offset):
eta, omega = latent_params
p_x, p_y, p_z = p
a_1, a_2, a_3 = a
epsilon_1, epsilon_2 = epsilon
x_0, y_0, z_0 = offset
return np.sqrt((p_x - a_1*get_pow_geometric(np.cos, eta, epsilon_1)*get_pow_geometric(np.cos, omega, epsilon_2))**2 + (p_y - a_2*get_pow_geometric(np.cos, eta, epsilon_1)*get_pow_geometric(np.sin, omega, epsilon_2))**2 + (p_z - a_3*get_pow_geometric(np.sin, eta, epsilon_1))**2)

p = [0.9999999921748904, 0.9999999999999998, -2.0]
a_true = [1, 2, 3]
epsilon_true = [0.5, 0.5]
offset_true = [0, 0, 0]
optimization_result = optimize.minimize(get_distance, [0, 0], args=(p, a_true, epsilon_true, offset_true))
eta_optim, omega_optim = optimization_result.x
``````

eta_optim and omega_optim values are -0.4562815152391673 and 0.07042550187811288 respectively; the distance is 0.0717705948348535

Then I tried to use the optimizer the following way:

``````p = torch.tensor([x2[0], y2[0], z2[0]])
epsilon_true = torch.tensor(epsilon_true)
a_true = torch.tensor(a_true)
offset_true = torch.tensor(offset_true)

def get_pow_geometric(f, angle, power):

def get_distance2(p, a, epsilon, offset):
p_x, p_y, p_z = p
a_1, a_2, a_3 = a
epsilon_1, epsilon_2 = epsilon
x_0, y_0, z_0 = offset
for i in range(30000):
d = torch.sqrt((p_x - a_1*get_pow_geometric(torch.cos, x[0], epsilon_1)*get_pow_geometric(torch.cos, x[1], epsilon_2))**2 + (p_y - a_2*get_pow_geometric(torch.cos, x[0], epsilon_1)*get_pow_geometric(torch.sin, x[1], epsilon_2))**2 + (p_z - a_3*get_pow_geometric(torch.sin, x[0], epsilon_1))**2)
d.backward(retain_graph=True)
optimizer.step()
print(x, d)
``````

While running get_distance2(p, a_true, epsilon_true, offset_true), I get

``````tensor([0.1000, 0.1000], requires_grad=True) tensor(2.9710, dtype=torch.float64, grad_fn=<SqrtBackward>)
....
....
....
``````

I would appreciate if anyone suggested how to fix the code in order to be able to get the same results as in the case of using scipy.optimize.

Below is the image, showing the function and the solution (obtained with scipy.optimize) - The function doesnâ€™t seem to be that complex so I donâ€™t know why the optimization fails.

Hi Niko!

Your pytorch code raises, in effect, `f (angle)` to a power, while your
numpy code raises `np.abs (f (angle))` to a power. When `f (angle)`
is negative and `power` is fractional, your pytorch code will return `nan`.

(As an aside, pytorch also supports the â€ś**â€ť syntax for exponentiation.)

Consider:

``````>>> import torch
>>> torch.__version__
'1.7.1'
>>> def get_pow_geometric (f, angle, power):
...
>>> def get_pow_geometric_B (f, angle, power):
...
>>> def get_pow_geometric_C (f, angle, power):
...
>>> get_pow_geometric (torch.sin, torch.tensor ([1.0]), 0.5)
tensor([0.9173])
>>> get_pow_geometric (torch.sin, torch.tensor ([-1.0]), 0.5)
tensor([nan])
>>> get_pow_geometric_B (torch.sin, torch.tensor ([-1.0]), 0.5)
tensor([-0.9173])
>>> get_pow_geometric_C (torch.sin, torch.tensor ([-1.0]), 0.5)
tensor([-0.9173])
``````

Best.

K. Frank

1 Like