How does one create a data set in pytorch and save it into a file to later be used?

[Brando_Miranda (MirandaAgent)]

Using Artur’s solution:

Train on a fraction of the data set

I want to train with SGD on say 10% of cifar10. However, I want that 10% to be the same fraction of cifar10 and not change once training has started. How does one do this? The only way I’ve thought of doing this was loop through all the data of cifar10 and once I’ve extracted the first 10% of the data, save it in a numpy file and then use that for training later and load that with a different dataloader. Would that work? What do people think?

Just sort the indices according to the criteron I have, save those indices and recover them from a file whenever I need them. Then use the data sampler!