# Constraining parameter vector to unit sphere

**URL:** <https://discuss.pytorch.org/t/constraining-parameter-vector-to-unit-sphere/206460>\
**Category:** Uncategorized\
**Created:** [July 17, 2024, 6:16pm UTC](https://discuss.pytorch.org/t/constraining-parameter-vector-to-unit-sphere/206460 "2024-07-17T18:16:22Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dherrera1911](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dherrera1911/32/61905_2.png) [@dherrera1911](https://discuss.pytorch.org/u/dherrera1911)\
**Post date:** [July 17, 2024, 6:16pm UTC](https://discuss.pytorch.org/t/constraining-parameter-vector-to-unit-sphere/206460/1 "2024-07-17T18:16:22Z")

</div>

I am creating a class for my model using nn.Module. It’s a simple model with only a set of linear filters as parameters. The parameters are `self.f`, defined by `self.f = nn.Parameter(f_init)` where `self.f` has shape `(n_filters, n_dims)`.

The issue is that I want to constrain the filters to have unit norm. So far I am doing this with the wonderful `geotorch` [package](https://github.com/lezcano/geotorch), using the call `geotorch.sphere(self, "f")`. Then I don’t have to think of this constraint when doing back-propagation.

My question is whether there is a simple way of implementing this constraint without `geotorch`, since it seems simple enough, and to avoid an extra required package that is not in pip. Naively, it occurs to me that I could just divide `self.f` by its norm after every update step. It is not obvious to me though whether this would mess with my gradient and learning somehow, although I don’t see why they should.

So, does it sound right to impose the unit-length constrain by dividing the vectors after each update step?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [July 17, 2024, 11:42pm UTC](https://discuss.pytorch.org/t/constraining-parameter-vector-to-unit-sphere/206460/2 "2024-07-17T23:42:41Z")

</div>

A quick check of the code indicates [`torch.nn.utils.parametrize.register_parametrization`](https://pytorch.org/docs/stable/generated/torch.nn.utils.parametrize.register_parametrization.html) is used which is then registering [`Sphere`](https://github.com/lezcano/geotorch/blob/ba38d406c245d609fee4b4dac3f6427bf6d73a8e/geotorch/sphere.py#L109) on the parameter, so you could try to use the same approach.

---

<div class="post-metadata">

**Author:** ![dherrera1911](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dherrera1911/32/61905_2.png) [@dherrera1911](https://discuss.pytorch.org/u/dherrera1911)\
**Post date:** [August 28, 2024, 6:24pm UTC](https://discuss.pytorch.org/t/constraining-parameter-vector-to-unit-sphere/206460/3 "2024-08-28T18:24:19Z")

</div>

I solved the problem using the parametrizations tool in Pytorch. I wrote a blogpost about how you’d implement the unit vector constrain I mentioned, and also how to constrain a matrix to be symmetric positive definite [Easy constrained optimization in Pytorch with Parametrizations - Daniel Herrera-Esposito](https://dherrera1911.github.io/posts/2024/08/constrained-learning-with-pytorch-parametrizations/)

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [August 29, 2024, 2:28pm UTC](https://discuss.pytorch.org/t/constraining-parameter-vector-to-unit-sphere/206460/4 "2024-08-29T14:28:24Z")

</div>

Really cool post! Thanks for sharing it here.