If you need this workflow to avoid the JSON issue, you could use a SubsetRandomSampler for each DataLoader, so that each DataLoader only loads a subset of the data.
However, I’m not sure how train is used.
Are you training 8 models simultaneously?