Given a training process I need, at each step, to get all gradients tensors associated to each individual sample of the batch; then I need to perform some operation on each of this gradients, finally collect them together and perform the
I found that Opacus could fit my problem, but:
I don’t need nor the gradient clipping neither the noise adding
on the other hand I need to access the set of gradients associated to each batch’s element and perform some operation on them; after these operations I’ll get a single gradient that will be used for the weight update (
Is it possible to do this with Opacus library?