Detect param update

autograd
1

I am trying to implement a mechanism which needs to be detected when parameters are updated, there are three ways to implement it:

  1. use register_backward_hook (problematic, it will not work sometimes, proved by experiment and statement in doc.
  2. use register_hook on every parameter (usable, but will not be executed right after parameter update)
  3. find a way to hook onto inplace operations such as add_, copy_ and addcdiv_, but I am not sure whether inplace operations are gauranteed to be used when you want to update a parameter (using param = some_modify_func(param) is possiple). And is there any clean way to hook onto them whithout creating a wrapper and wrap parameters inside ?

What’s your suggestion? Thanks in advance!

2

Hi,

You can definitely use register_hook to know when gradients are computed for a Tensor (it is executed just before the gradient is accumulated and the new value is passed as an argument).
But I don’t think you will be able to have a callback every time a Tensor is modified inplace :confused:

3

Thank you! Eventually I chose the second option, currently there is no better option than register_hook :confused: