Ctx.needs_input_grad behaviour

autograd
12

I just put my fears into the code and it seems to be working anyway. What I did:

  1. Introduced dumb var which is doing nothing at all as one of the inputs: ctx.save_for_backward(dumb, weight, input , bias) and changed return in a backward accordingly.
  2. Changed position of variables.
  3. Deleted calculation of gradients for input case.
    And everything is working perfectly:
ctx.needs_input_grad (False, True, True, True) - look at the 3rd pos it is an input.
ctx.needs_input_grad (False, True, False, True) - look at the 3rd pos it is an input.

I mean logic is completely broken in backward but I can conclude that PyTorch knows x that it is an input data and doing everything perfectly and it is not related to position of input and whether backward implemented correctly although it checks output shape of the gradient for correctness.

I can sleep well. Dear, @albanD, thank you very much for help! Have a very productive day!

Andrei

1 Like