Hi everyone !
My problem is that I would like to extend “accumulate_grad.cpp”
in order that all the variables p which require grad have a new field
called (say) p.vargrad which will be identical in shape to p.grad
which - if I understand correctly - is the mean of the gradient
on a mini-batch, but containing the variance of the gradient on
a mini-batch.
To be clear p.vargrad.data will be a matrix in which each entry is the
variance of the parameter p on a minibatch and parallels
exactly p.grad.data.
Is it possible to change the code and recompile pytorch
in order to do this ?
If yes an someone help me to do this ?
Thanks in advance !!!