I want to change the saved input to a module that is used to calculate the gradients in the backward call.
To be clear, let’s say I want AdaptiveAveragePool2d computes the gradients given an input2 rather than the input that passed through it during forward call.
One would write a custom autograd function where
input2 is saved in ctx instead of input. But I was wondering if there is a workaround? Implementing backward function of AdaptiveAvgPool from scratch has proven tricky.