Hi,
Not sure how to do that exactly, but I would suggest to use a custom Sampler based on the SequentialSampler that is used by default when shuffle=False.
In particular, you can create one that will return return only part of the indices. Then recreate a dataloader with the regular sampler for the rest of the training.