I would like to define a backward hook function that computes per-example gradients (to compute some other statistics). It looks like register_full_backward_hook has hook functions that have input args (module, grad_input, grad_output) but no ctx. Would there be any way to access the tensors saved in ctx from the hook so I can compute the gradients as desired?
I don’t believe this is possible but you could write a custom autograd.Function which would allow you to store intermediates and use them in the backward. Depending on what you want to store you might need to re-implement the forward computation, too.
Okay got you. That was what I was thinking too but I just wanted to check if doing a separate forward pass each time would be necessary. Thanks!