How i can find best learning rate?

I waste so much time to find best learning rate for model. Is there any fast way other than random vars, i usually look loss graph to adjust learning rate.

You can:
1/ Use the adaptive learning rate optimization method such as ADAM family
2/ A dynamic way to fine-tune learning rate while training (example: cycle learning rate: https://arxiv.org/abs/1506.01186)
3/ Switch the optimization while training with the best learning rate (https://arxiv.org/abs/1712.07628)

1 Like

Thanks for help. Iā€™m going to read papers.