I waste so much time to find best learning rate for model. Is there any fast way other than random vars, i usually look loss graph to adjust learning rate.
1/ Use the adaptive learning rate optimization method such as ADAM family
2/ A dynamic way to fine-tune learning rate while training (example: cycle learning rate: https://arxiv.org/abs/1506.01186)
3/ Switch the optimization while training with the best learning rate (https://arxiv.org/abs/1712.07628)
Thanks for help. I’m going to read papers.