Handling the hidden state with minibatches in a RNN for language modelling

No, the sentences in a batch are processed independently and in parallel in one go. Otherwise there would be no need for the batch_size dimension in the shape of the hidden state.