I need to train model on new instance and find performance or loss of model over the validation dataset.
First I would like to train model on training dataset, then in loop it needs to retrain on new instance and evaluated on validation set in order to see which item provides better result when it contributes in training set.
Just making sure I understand: you want to train a model architecture twice (or multiple times), then evaluate both on the evaluation set and select the instance that performed best for the next epoch? So that you can optimize the randomness in weights initialization and other possible random aspects like dropout?
In that case, you can simpy train your two separate instances by having differently named optimizer and model variables… However, I suspect that you would need two times (or N times) as much memory on your GPU(s) as for one model, since you keep multiple instances of the weights etc. That is probably why saving and loading is the best approach, so that you can maximize your batch size and GPU usage…
Thanks for the reply!
Let me make it clear. It is kind of active learning. I train the model (A) over training dataset (D) and I want to add new instance to the training dataset to retrain it again!
To do that, I need to retrain the A over each instance from other dataset (B) and evaluate on the validation dataset to choose the best one in order to put it in the training dataset (D).