Writing my own derivative formula

autograd
1

Hi,

Is there any way to supply a derivative formula to the Autograd engine? I want to build a network and compute gradients using a custom formula. Any help are greatly appreciated!

2

https://pytorch.org/tutorials/beginner/examples_autograd/two_layer_net_custom_function.html
You just have to overwrite the backward function in the nn.Module

3

Note that the backward function is not on the nn.Module but on an autograd.Function!
You have more details here as well: Extending PyTorch — PyTorch 1.8.1 documentation

4

Hups! Have to say never tried it :slight_smile: thanks for the clarification.