I have an algorithm I’d like to implement that uses backprop but does not actually pass the gradient information backwards instead passes something approximating a function inverse. The point is, I don’t want to use the gradient information but will use other information in the backprop process. Is it possible for me to redefine all the gradients to use the information I want? I should add, I know how to compute what I want from the input / output and function, I just need to pass the resulting values as “gradients” instead of what pytorch uses.
I think your best bet would be to write a custom pytorch hook for this.
For more details you can refer to this.
The main idea would be to register the hook to your parameters and then with the custom logic, backpropagate the values you want to.
I would also like to see if the community has a better way to go around it.
1 Like