Non-autodiff-computationally tractable gradients in forward pass

Hi -

Dumb question! May be I haven’t encountered it in the docs.

Suppose due to transformation and some bad functional form, I’m not able to build an expression graph in the forward pass. However, I’m able to do some analytic calculations and the gradient for the ill-formed loss function is tractable.

Is there a way to substitute grad or gradients WRT a loss function, with something I’ve implemented, for example, using finite differencing?

I’m sure my deep learning understanding is rather poor.




You can check this section of the doc that explains how to specify the backward formula for a forward that is not differentiable.