 # How might autograd differentiate complex functions that are not complex-differentiable?

Hello Forum!

There are useful functions of a complex variable, `z`, that are not
“complex-differentiable,” that is, they are not analytic functions.
Two examples would be `z.conj()` and `|z|**2 = z * z.conj()`.

Note that these functions are differentiable when viewed as real
functions of two real variables, `x` and `y`, where `z = x + yj`.

I think that when a function is analytic, autograd should return
its conventional complex derivative. However, we would like
autograd also to do something “useful” for non-analytic functions
that are differentiable when understood as real functions.

For example, it would be nice if gradient descent would work for
minimizing `|z - z0|**2 = (z - z0) * (z - z0).conj()` with
respect to `z` (which takes on its minimum value of `0` when `z = z0`).

I don’t have a well-thought-out proposal for how to do this. But I
came across this exposition that discusses some of the core issues:

The Complex Gradient Operator and the CR-Calculus

One facet of this issue is illustrated in this post where naive gradient
descent is shown to fail for `|z|` (for complex `z`) using autograd in
version 1.6.0:

Best.

K. Frank

2 Likes

Hi Frank,

We actually had a very long discussion about this exact question and what to do.
It is also quite fun to see that jax and Tensorflow currently define two different gradients for these non-analytical functions!
You can see the whole discussion here: Follow the JAX or Tensorflow convention for meaning of grad() with complex inputs · Issue #41857 · pytorch/pytorch · GitHub
Also you can find in our doc a fairly extensive summary of the result of that discussion: Autograd mechanics — PyTorch master documentation

Happy to discuss it more if you’re interested!