Using model.[layer_name].weight.grad.data I am able to access the gradients for weights of a particular layer. However, I assume that these gradients are averaged across samples in a mini-batch. Is there a way for me to access the gradients of weights for each sample?
I was able to obtain per-sample gradients for activations or neurons using register_hook, so not sure what to do about weights.