Assume that we have a pertained NN model like LeNet-5 which successfully predicts handwritten digits. In this case number of features is 784 (assuming 28x28 input images) and number of outputs is 10. Sum of the output values(probs) adds up to 1 and each output shows the probability of that class for the given input image.
Now, assuming that I already have this model. And I can predict the class of any input image using feedforward model prediction.
My question is: For any test input image, I would like to calculate derivative of any model output w.r.t model input features. This way I will be calculating a 784x10 Jacobian matrix J.
For example J[0,0] is the derivative of output 0 w.r.t input feature 0 and J[2,3] is derivative of output 3 w.r.t input feature 2 and so on…
Actually maybe I can calculate each element of this Jacobian via a messy code block but I wonder if there is any easy and elegant way of doing so.
Any comment on how to calculate the J matrix?