Hello,
I’m working on a shape deformation library (IMODAL on github). At some point, we optimize a differentiable function. LBFGS works best compared to SGD-variants (up to our knowledge).
However, we wish to prioritize some variables over some others. First idea was to change the learning rate for the desired priority variables.
At the time of writing this question, LBFGS from torch.optim does not support per-parameter options.
Is there a workaround to achieve this? What are other optimization options I can reach out for given that LBFGS worked better than SGD and Adam. I’d appreciate any links to resources providing insight on LBFGS vs. SGD-variants and whether we can ‘‘emulate’’ LBFGS with a first-order method.
Thanks in advance,