# Optimization problem

**URL:** <https://discuss.pytorch.org/t/optimization-problem/225631>\
**Category:** Uncategorized\
**Created:** [October 23, 2026, 9:31pm UTC](https://discuss.pytorch.org/t/optimization-problem/225631 "2026-10-23T21:31:36Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ghostv](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/7ea924/32.png) [@Ghostv](https://discuss.pytorch.org/u/Ghostv)\
**Post date:** [October 23, 2026, 9:31pm UTC](https://discuss.pytorch.org/t/optimization-problem/225631/1 "2026-10-23T21:31:36Z")

</div>

Hi!

I’m totally new to using torch for solving optimization problems, but for what i understand, i can minimize a loss function after specifying which variables the optimizer can modify.

Now, i wanna ask if the kind-of-problem i need to solve can be solved using this approach (because the function i need to minimize is kinda complex), or if the loss functions can only be used for simple polynomial equations.

The problem is, i have:

- A fixed 3D cloud of points → An (n,3) shaped tensor (let’s call it x1)
- A variable 3D cloud of points → An (m,3) shaped tensor (let’s call it x2)

1. Now, i get the distance between every pair → An (n,m) shaped cdist(x1,x2) tensor (let’s call it x3)
2. Then i take the min value per row → An (n,1) shaped min(x3,dim=1) tensor (let’s call it x4)
3. Then i threshold it to a d value → An (n,1) shaped where(x4 \<= d, 0, 1) tensor (let’s call it x5)

Now, my idea is to minimize x5.sum(), which is basically optimizing the values of x2 so it maximizes the amount of x1 points that end up being closer-than-d to the x2 positions.

My question is, can this be achieved defining a loss function and optimizing it? Like.. making a function with all these steps (ie., def bunch\_of\_all\_those\_calculus()) and then just passing that function to the loss?

Any help is welcome!

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![valerian.rey](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/valerian.rey/32/74283_2.png) [@valerian.rey](https://discuss.pytorch.org/u/valerian.rey)\
**Post date:** [October 24, 2026, 4:29am UTC](https://discuss.pytorch.org/t/optimization-problem/225631/2 "2026-10-24T04:29:19Z")

</div>

Hi! PyTorch is mostly made for gradient-based optimization, which could be used on your problem (although it’s a non-convex problem so I think you don’t have any guarantee other than reaching a local minimum given the appropriate LR).

That being said, to compute gradients, the function must be differentiable. And when I say “the function”, I mean the function taking as input x2, and returning x5.sum(). You basically want to compute the gradient of x5.sum() with respect to x2, and use that to update x2 with gradient descent.

In your case, x3 can be differentiated with respect to x2, and even x4 with respect to x4, but x5 is non-differentiable with respect to its condition. What you could do is minimize a differentiable approximation of x5 instead, by using a soft threshold function for x5 (e.g. a sigmoid, shifted by d).

Something like:

```python
x5 = torch.sigmoid(x4 - d)

```

The full thing would be something like:

```auto
x1 = ...
x2 = torch.randn(..., requires_grad=True)
optimizer = torch.optim.SGD([x2], lr=...)

for ...:
    x3 = torch.cdist(x1, x2)
    x4 = torch.min(x3, dim=1)
    x5 = torch.sigmoid(x4 - d)
    loss = x5.sum()
    loss.backward() # Compute gradient of loss wrt x2, store it in x2.grad
    optimizer.step() # Update x2
    optimizer.zero_grad() # Free x2.grad

```

---

<div class="post-metadata">

**Author:** ![Ghostv](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/7ea924/32.png) [@Ghostv](https://discuss.pytorch.org/u/Ghostv)\
**Post date:** [October 24, 2026, 10:23am UTC](https://discuss.pytorch.org/t/optimization-problem/225631/3 "2026-10-24T10:23:02Z")

</div>

Perfect, thanks a lot!

I’ll give it a try in a while, but it clarifies it perfectly; so, basically, all the expression must be differentiable with respect to the ind. variable.

Thanks again mate!

---

<div class="post-metadata">

**Author:** ![Ghostv](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/7ea924/32.png) [@Ghostv](https://discuss.pytorch.org/u/Ghostv)\
**Post date:** [October 24, 2026, 6:49pm UTC](https://discuss.pytorch.org/t/optimization-problem/225631/4 "2026-10-24T18:49:25Z")

</div>

One extra question… Is it possible to narrow the domain in which the optimization variable (x2) moves? Ie. like.., a \< x2[:,0] \< b or to just specify set of values it could take? From your answer, i think i can just kinda “force” the intervals with a bunch of sigmoids, but seems sloppy.. is there any built way to achieve that?

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![valerian.rey](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/valerian.rey/32/74283_2.png) [@valerian.rey](https://discuss.pytorch.org/u/valerian.rey)\
**Post date:** [October 25, 2026, 5:03am UTC](https://discuss.pytorch.org/t/optimization-problem/225631/5 "2026-10-25T05:03:36Z")

</div>

> One extra question… Is it possible to narrow the domain in which the optimization variable (x2) moves? Ie. like.., a \< x2[:,0] \< b or to just specify set of values it could take?

This would be some form of constrained optimization. You could clip the values of x2 after each update (see torch.clip) to constrain it to be in the correct interval. This is called projected gradient descent, because x2 is projected back onto the feasible set of solutions at each step. See [this](https://discuss.pytorch.org/t/how-to-do-constrained-optimization-in-pytorch/60122) for example.

> From your answer, i think i can just kinda “force” the intervals with a bunch of sigmoids, but seems sloppy..

I’m not sure to understand what you mean by that, but another idea in that direction would be to optimize some other variable, say x2\_prime, without constraint (so from -inf to +inf), and have x2 = a + sigmoid(x2\_prime) \* (b - a). This way, x2 will by construction be between a and b, and you won’t have to do any clipping. Note that if you do that, you’ll have to set x2\_prime instead of x2 as the variable to optimize when declaring your optimizer.

Lastly, you could relax the constraint into a regularization term, by penalizing how far x2 is away from the interval [a, b], and adding that to the loss function. This way, you won’t enforce that x2 is in [a, b], but you would push it in that direction.

There’s probably other ideas that could work too.

> is there any built way to achieve that?

Not that I know of

---

<div class="post-metadata">

**Author:** ![Ghostv](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/g/7ea924/32.png) [@Ghostv](https://discuss.pytorch.org/u/Ghostv)\
**Post date:** [October 25, 2026, 3:31pm UTC](https://discuss.pytorch.org/t/optimization-problem/225631/6 "2026-10-25T15:31:06Z")

</div>

Yeah, what i have in mind was basically the same as what clip/clamp does, to trim-to-zero the values outside the interval.

Well, this gives me a good input to start testing; thanks a real lot mate!

---

<div class="post-metadata">

**Author:** ![drezaptmp](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/drezaptmp/32/80182_2.png) [@drezaptmp](https://discuss.pytorch.org/u/drezaptmp)\
**Post date:** [October 26, 2026, 2:19pm UTC](https://discuss.pytorch.org/t/optimization-problem/225631/7 "2026-10-26T14:19:39Z")

</div>

Can you define the function you’re trying to approximate rigorously mathematically? Then I will consider. In solving optimization problems, in professional, academic literature, we first have to rigorously define the problem we are trying to solve, or else we get this, “mathematical,” garden of forking paths problem.