Suppose I have a very large dataset, so large that it would take a considerable time to go through them even once. I’m training a model on this dataset and at some point before even completing one epoch on the whole dataset, my training might be stopped or killed. I want to be able to resume the training process. The problem is, I don’t want to start going through the dataset from start, i.e. I want to resume the training from the exact last sample that the model was trained on.
At the moment, I can’t figure out how to set the index for Dataloader object.
Is there a way to do that?