Custom autograd.Function: must it be static?

Hi,

Option (1) is the old way to define Functions. This does not support gradients of gradients and it’s support might be discontinued in the future (not sure when).
The second one is the way to go. Note that you can do exactly the same thing as you can save arbitrary stuff in the ctx (the same way you would save in self in (1)), and the apply method that calls forward accept any parameter, so you can just pass what you used to give the __init__() function here. That means that you don’t need to define options globally, just pass them to the forward method.

7 Likes