Gradient of output wrt specific inputs

autograd
1

Hi there,

I am trying to retrieve the gradients of the output variables wrt the input variables of a Neural Network model (loaded from a .pt file). Note that the neural network describes a dynamical system in which a history of states is taken into account. The network is too complicated to have an intuition of the range in which the gradients should lie.

I am currently tackling it the following way:

    x = torch.zeros(1,1500) # 1500 inputs to model
    x[0][:] = torch.from_numpy(array_of_inputs)  # Give all input and state data we have to the NN
    x.requires_grad = True  # Make sure gradients can be extracted

    # Compute outputs
    out = model(x)
    out = out[0]

    alphadot = out[1]  # Get gradient of variable alphadot (index in output array is 1)
    gradient = torch.autograd.grad(outputs=alphadot, inputs=x, grad_outputs=torch.ones_like(alphadot),
                                   retain_graph=True)
    gradient = gradient[0].tolist()
    gradient = gradient[0]

The variable gradients contains the gradients of output with index 1 (alphadot) with respect to all 1500 input variables in x. Does the code snippet above behave as I think it does in my description? If not, what am I doing wrong?

Thanks a lot for your help!

1 Like
2

Hi,

Yes this code does what you think :slight_smile:

1 Like
3

Thanks for the confirmation!

4

What if we want to store the gradient of output w.r.t input for every epoch? Does this code work too? why in this code there is no loss.backward()?

5

There is no notion of epoch nor loss is OP’s example.
Checkout Neural Networks — PyTorch Tutorials 1.9.0+cu102 documentation for an intro to how neural networks with PyTorch.