Is there any optimizer in pytorch that currently supports line search? I want to solve an optimization problem where being able to guarantee that the error always goes down is necessary. Last time I checked, LBFGS for example, in theory supports line search, but it’s not implemented yet in pytorch. Is there any optimizer that supports line search?
1 Like
I would also be very interested in this. Have you made any progress?
Making a shameless self plug, but if you’re still interested in this, I’ve just made public my implementation of L-BFGS which contains both a backtracking line search (with or without cubic interpolation) and weak Wolfe line search. Take a look here: https://github.com/hjmshi/PyTorch-LBFGS. Any feedback would be appreciated!
2 Likes
FYI, An implementation of L-BFGS with Strong Wolfe line search is merged in https://github.com/pytorch/pytorch/pull/8824
1 Like