I find some codes in this and become curious about the function of @once_differentiable
.
2 Likes
If you wrap your Function
's backward
method with this wrapper, the Tensors that you will get as input will never require gradients and you don’t have to write a backward function that computes the gradients in a differentiable manner. For example, you can use other libraries to do the computation.
If you try to backward through the backward pass of such Function, an error will be raised stating that this Function is only differentiable once.
1 Like
Thx for your reply but it’s a bit confusing…Is that meaning we cannot compute high-order gradients from this function?
Yes.
You should use it when you write a backward that cannot be used for high-order gradients.
4 Likes
Got it!
Very helpful!