Hi! I am trying to implement an efficient dataloader which would yield a minibatch that includes the same sample which undergoes different preprocessing methods. e.g. a single minibatch with size 8 should include: [sample n preprocessed with filter 1, sample n preprocessed with filter 2, … , sample n preprocessed with filter 8]
Next, I want to do the opposite, to have a dataloader which would yield a minibatch with one preprocessing with multiple samples, and the next minibatch would include another preprocessing method.
To speed things up even more, I have processed all my samples with all 8 filters and saved them individually, now they just need to be loaded. But how can I control the loading of the data at the minibatch level, and still ensure parallel loading? Thanks