I’m trying to dump all intermediate modules’ input and output tensor values both during forward and backward. So I just simply iterate through the named_modules, and register_forward_hook and register_full_backward_hook as usual. But to my surprise, in my model everything works fine except for LSTM module. I only see the forward hook gets called for this LSTM module, the backward hook function never gets called at all for it.
But if I change register_full_backward_hook to register_backward_hook, I can see the hook function gets called. I thought register_full_backward_hook is better than register_backward_hook, since the latter might just dump the intermediate results in a module. But why for LSTM, only the latter works here?
Is there anything wrong with LSTM backward hooks?
Hi @BruceDai003,
Can you share a minimal reproducible script? Also, register_backward_hook gives incorrect (in most cases) results so it’s best not to use it!