Hello, users of pytorch. The reason that I want to write my own activation function with cuda C++ is that I want to calculate my own backpropagation. I tried to find some tutorials but very limited resources can be found. A tutorial here Custom C++ and CUDA Extensions — PyTorch Tutorials 1.10.0+cu102 documentation is useful. I think some modifications of present files of an activation function would be the easiest way for me to write my own one. Could someone please tell me where should I start or suggest other resources for my request? Thank you in advance.
autograd examples perhaps (NB: “activation” characteristic doesn’t imply anything for coding or autograd, you need just “a function”).
Thank you for this resources. I am trying to figure it out. The resources use auto gradient. Is there any tutorials that use no auto gradient and write their own backward functions?
have you scrolled down to “Inherit from Function” section? that’s it - subclassing the Function class, almost the same as in Python.