Gradient of output wrt specific inputs

Hi there,

I am trying to retrieve the gradients of the output variables wrt the input variables of a Neural Network model (loaded from a .pt file). Note that the neural network describes a dynamical system in which a history of states is taken into account. The network is too complicated to have an intuition of the range in which the gradients should lie.

I am currently tackling it the following way:

    x = torch.zeros(1,1500) # 1500 inputs to model
    x[0][:] = torch.from_numpy(array_of_inputs)  # Give all input and state data we have to the NN
    x.requires_grad = True  # Make sure gradients can be extracted

    # Compute outputs
    out = model(x)
    out = out[0]

    alphadot = out[1]  # Get gradient of variable alphadot (index in output array is 1)
    gradient = torch.autograd.grad(outputs=alphadot, inputs=x, grad_outputs=torch.ones_like(alphadot),
    gradient = gradient[0].tolist()
    gradient = gradient[0]

The variable gradients contains the gradients of output with index 1 (alphadot) with respect to all 1500 input variables in x. Does the code snippet above behave as I think it does in my description? If not, what am I doing wrong?

Thanks a lot for your help!

1 Like


Yes this code does what you think :slight_smile:

1 Like

Thanks for the confirmation!

What if we want to store the gradient of output w.r.t input for every epoch? Does this code work too? why in this code there is no loss.backward()?

There is no notion of epoch nor loss is OP’s example.
Checkout Neural Networks — PyTorch Tutorials 1.9.0+cu102 documentation for an intro to how neural networks with PyTorch.