Getting individual backward() info on a loss function with (reduction='none')

[tom (Thomas V)]

I think you want something like discussed in this thread (see also the bug report):

Efficient computation of per-sample examples

Additional discussion of [feature request] Simple and Efficient way to get gradients of each element of a sum @tom I hope I interpreted your last post correctly by posting here. The code to reproduce the numbers I see is available here. The main.py file runs different ways of computing individual gradients and checks/times them. Your recommendation seems to be along the line of Goodfellow’s derivation, implemented as goodfellow, timing output under goodf. I don’t think it is a bug per-se, …

It’s not something that you would change at the “loss end” of the computational graph, but at the parameter end.

Best regards

Thomas