Dear all,
my loss function involves a term f(x) from which I want to compute part of the derivative by hand making use of the chain rule, i.e. df(x)/dw = df(x)/dx * dx/dw. The reason is that I have an explicit and pre-computable expression for df(x)/dx. The code is as follows
Tensor x;
Tensor f = compute_f(x);
Tensor df = compute_df_fx(x);
How can I make sure that the df term is used in the back propagation step?
Hi Matthias!
Package compute_f() and compute_df_fx() as a custom autograd.Function.
Roughly speaking compute_f() will become your Function’s forward()
method, and compute_df_fx() will become backward().
If it is more efficient or more convenient for your use case, you can compute
compute_df_fx() in the forward pass (in forward(), store in ctx (the context),
and then use it in the backward pass (in backward()).
Best.
K. Frank
Hi K. Frank,
Thanks very much for your suggestion. Meanwhile, I also came across another solution here which seems easier. Do you know if Variables are supported in C++?