How is sequencing established in pytorch RNNs?

From what I understand, unlike feedforward nerual networks, the sequence of data is really important in RNNs because prediction and data from observation 1 of X is used in the prediction of subsequent rows.

But in tutorials I’ve seen, the forward function in RNNs still seem to operate on the whole training matrix X like it’s about to do everythign in parallel.

So I tried to create an empty list to hold all the predictions for the calculation of the loss as well as a loop to iterate through the rows in order like this:

    def forward(self, X0):
        predictions = []
        for i in range(X0.shape[0]):

But doing things this way, the predictions are disconnected from the parameters and the process of backpropogation will not work. So my questions are, how should I design my model to process each row of the training matrix in order from top to bottom if not with a loop and how should the predictions in each RNN cell be recorded?