# Backward pass for hyperbolic tangent?

Hello I am trying to build a custom loss function which include using tanh.
if forward pass is y=tanhx
than how should be the backward pass?
In other word what should be the relationship between gradOutput and gradInput?

I guess you’re implementing the backward by hand? Otherwise it will work with the autograd.

The derivative for tanh is quite common and can be found here for example. The C implementation of the backward pass by pytorch is here.

@albanD Thanks for the reply. I’m a bit confused by the autograd process. The document says it computes the vector-Jacobian result, which usually returns a vector.

However, the tanh function is element-wise, that is, Y_{ij} = nn.tanh(X_{ij}), where X and Y are both matrices.

In this case, we have a matrix-valued function and the gradient of Y wrt X would be a 4-D Jacobian matrix.

The C-souce code is like this:
`*gradInput_data = *gradOutput_data * (1. - z*z);`
It seems so perform an elementwise multiplication: Y_ij * (1-X_ij^2)

Here, `(1. - z*z)` is actually the “diagonal” of the jacobian and the result is computed by doing an element-wise product with the gradient flowing back to get the gradient wrt the input.