How to implement armijo step size in a neural network developed by Pytorch? Currently the neural network is using optim.Adam as the step size.
There is a nice github repo (GitHub - hjmshi/PyTorch-LBFGS: A PyTorch implementation of L-BFGS.) that implements L-BFGS optimization method for Pytorch and allows for different kinds of line search, for example, arimijo.
There they have the armijo example.
Please refere to that.