How to sample a batch from a different subset every iteration

I have a dataset with many subsets like [[1,2,3,4,5],[6,7,8,9,10],[10,11,12,13,14],[15,16,17,18,19,20]]
for every mini batch in every iteration, I want to sample randomly from a random subset.
eg, if batch_size = 3, a mini batch could be [1,4,5] or [10,13,14]
I looked at the samplers provided in pytorch, but it seems none of them can do this.