Hi all,
I need to implement double backward for 2D max pooling.
I’ve been trying to understand how some of the other ops were converted, without much success.
Can anyone provide any insights?
e.g. A rough explanation of how the Linear layer was converted might be really helpful (what is ctx??)
Thanks.
I found this quite helpful for understanding the Autograd refactor and how the new function defs work: https://github.com/pytorch/pytorch/pull/1016
However, for max-pooling I am still quite confused as it seems like the logic is implemented in the THNN backend…I cannot actually find the source code for this.
Should the pooling function be moved out of THNN in order to implement the higher order derivatives? Or is there a simple way to use the existing functions?
Found this comment in the Threshold function that was added to allow for ReLU double backward :
# TODO: This class should be removed once THNN function support Variable backward
So my understanding is that I need to write something similar for pooling, at least until THNN support is added.