I have a huge network (e.g. resnet)
I like to change every gradient for each function which is computed via back-propagation.
Let say I like to add a constant
C to it.
can someone help me to understand how via
aaply i can do it?
I think my function should be like this?
class Change(torch.autograd.Function): @staticmethod def forward(ctx, input,C): ctx.save_for_backward(input) return input @staticmethod def backward(ctx, grad_output): input, = ctx.saved_tensors # why we should do input, not intpu? grad_input = grad_output.clone() return grad_input+C
I am not sure where should i put/apply it though