How to partially and alternatively update the model?

I’ve designed a model with the weights in the network is supposed to be updated according to the input. I got confused when I tried to implement it with batch input. Since batches are processed in parrallel, I was expecting to process each sample accordingly. For example, suppose there are two RNNs named 1 and 2, when an input is passed to the model, it will pass through both RNNs for further calculation, but only one of the RNN’s weights should be updated through BP (either RNN1 or RNN2 in this case). How can I implement with PyTorch? Any suggestion or simple example could be helpful. Thank you.