What does the function wrapper @once_differentiable do?

I find some codes in this and become curious about the function of @once_differentiable.

2 Likes

If you wrap your Function's backward method with this wrapper, the Tensors that you will get as input will never require gradients and you don’t have to write a backward function that computes the gradients in a differentiable manner. For example, you can use other libraries to do the computation.
If you try to backward through the backward pass of such Function, an error will be raised stating that this Function is only differentiable once.

1 Like

Thx for your reply but it’s a bit confusing…Is that meaning we cannot compute high-order gradients from this function?

Yes.
You should use it when you write a backward that cannot be used for high-order gradients.

3 Likes

Got it!
Very helpful!