I’m not sure if I understood everything but if you create 2 dataloaders with shuffle=True, they’re both gonna get shuffled randomly and differently.
What you could try is set the random seeds to a fixed same value for both dataloaders using worker_init_fn as described here.
Also check this whole thread: Fix seed for data loader