Hi guys
I have a trained model which is intended exactly for my problem. I want to improve this model by training it further on new data. Now I have a question how to proceed in choosing the hyperparameters, especially the learning rate. Currently I am trying to train 1 epoch on all training data for a certain learning rate and then compare the validation of the models on the test data. Here I try to build up certain ranges e.g. from 1e-3 to 1e-4 and then I take the best final result and continue from there, if e.g. 1e-4 was better, then I would test 0.5e-4 to 1.5e-4 as the next range until I notice no or minimal changes at some point. Does this procedure make sense? An epoch lasts about 8 hours and the procedure therefore takes a very long time.
With real numbers i first tested
5E-3 and 1e-4, and 1e-4 was better
so i tested 1e-4 1e-5, 1e-5 won etc
Thanks