I have multiple datasets, I want to sample them with some weights like 2x dataset1 1x dataset2, … and then define a dataloader for this concatenated dataset. How I can define a custom dataset which can have sampling mechanism?
Also the datasets are very large, and I cannot bring them to memory. thanks
After making concateneted sampled dataset, I want to define 1 dataloader for it, also I need to make sure each batch comes from one type of dataset. thanks