Backpropagating Error Through Neural Network to Train Another Neural Network

Hello! I am reading Nguyen and Widrow’s 1990 paper “Neural networks for self-learning control systems.” I have a general understanding of the paper: a neural network (NN) is first trained to emulate an unknown plant. Then a second NN is trained to control the NN emulator.

I understand how to train the NN emulator. However, I don’t understand how to train the NN controller. In the subsection “Training the Neural Network Controller,” the paper states that we can backpropagate the plant error through the (trained) NN emulator to calculate the control error to train the NN controller. This sounds like we are reversing the NN emulator and inputting the plant error and outputting the control error.

Is there a PyTorch function for doing this? Or is there a GitHub repository that I can look at to see how this is done? Any help is greatly appreciated! Thank you so much for your time!