Altering tensor hook invocation order

autograd
1

Hey everyone,

I am in the process of extending captum’s lrp implementation by some of the rules that are not in the current pull request as I need those for a project at work. Due to that I have some questions on tensor hooks:
1.) It seems the invocation order of hooks on tensors is determined by the order of calling regsiter_hook(), which seems reasonable, but I could not find a definitive statement on this in the documentation. Can anyone clarify whether this is in fact the case?
2.) Is there a way to alter the invocation order after registration? I tried to alter the OrderedDict stored in the _backward_hooks attribute, but this does not work. So I assume the order is somehow baked into the computational graph? Is there any way to alter this apart from removing hooks and reattaching them in the desired order?

Best regards and thanks

2

It seems the invocation order of hooks on tensors is determined by the order of calling regsiter_hook()

Yes they are called in the order they were installed.

Is there a way to alter the invocation order after registration?

There is no official way to do this no, you will need to uninstall and reinstall them.
The way these hooks are implemented is a bit complex as it depends on where the Tensor is in the graph so I don’t think you can hack your way around that right now. Or you will have to read the code related to hooks in the engine but I don’t think we expose the internal structures to python anyways.

1 Like