Try to train a neural network's derivative function

I wonder anyone has tried training a NN’s derivative function using pytorch. I have seen someone is using tensorflow to do this
For example, in the supervise learning. The input and labels keep unchanged, the real output is got from the autograd from the neural network’s output to the neural network’s input?