I want to change the saved input to a module that is used to calculate the gradients in the backward call.
To be clear, let’s say I want AdaptiveAveragePool2d computes the gradients given an input2 rather than the input that passed through it during forward call.
One would write a custom autograd function where
input2 is saved in ctx instead of input. But I was wondering if there is a workaround? Implementing backward function of AdaptiveAvgPool from scratch has proven tricky.
I am afraid this is not really possible. This would compute the “wrong” gradients and so autograd will raise an error if you try to change the input inplace before computing the backward.
For the particular case of
AdaptiveAveragePool, the input value is not actually needed when you compute the backward (just its size). So I am not sure what you expect to happen if you modify it?
Ahh! You’re right! I miss the point that I don’t need values for AdaptiveAveragePool!
Good to know that there is no way to do this in general.