there is no way to access the gradients wrt weight for each individual sample. Gradients wrt weights in pytorch are always accumulated (even over the mini-batch). If you want gradients wrt each sample, you will have to run each sample individually through the network. This is because both the THNN backend and CuDNN dont support individual sample gradients wrt weight.