Unexpected (wrong?) result when calculating derivatives using autograd

If I allow substitutions, it becomes arbitrary as to what a derivative wrt a intermediate like z, e.g. df_dz could be.

Suppose instead I have z:= 2x + y, then there’s two ways of writing f:

  1. 2z - x
  2. z + x + y
    Would df_dz be 1 or 2 here? It isn’t well-defined.

A different way to think about this that is less confusing might be:

We have two functions, z, f defined as follows:
z(a, b) := a + b
f(w, x, y) := w + x + y

In each of the expressions above, the variables w, x, y, a, b are only meaningful within the scope of the function definition.

In the original example, I can compose the two functions and define g(a, b, x, y) := f(z(a, b), x, y)
and then h(x, y) := g(x, y, x, y).

From the perspective of f, it does not know could be related and hence the answer is always 1 without any ambiguity.

1 Like