This is my first time in these forums. Hence, please let me know if I could describe my issue with more clarity. I am only running on CPU right now, but will move on to powerful GPUs once I get it to work on CPU. I am using pytorch 1.6.0.
My intention is to use LBFGS in PyTorch to iteratively solve my non-linear inverse problem. I have a class for iteratively solving this problem. This class uses the LBFGS optimizer, specifically, with the following default parameters -
self.optimizer = torch.optim.LBFGS(self.model.parameters(), lr=1, max_iter=20, max_eval=None, tolerance_grad=1e-07, tolerance_change=1e-09, history_size=10, line_search_fn='strong_wolfe')
It has a method called “run” for performing the optimization, which is given below,
def run(self, data): data = torch.tensor(data, dtype=torch.double, requires_grad=False) def closure(): if torch.is_grad_enabled(): self.optimizer.zero_grad() data_pred = self.model() loss = self.loss_fn(data_pred, data) if loss.requires_grad: loss.backward() return loss loss = self.loss_fn(self.model(), data).detach().numpy() print('Init loss: ', loss) self.optimizer.step(closure) loss = self.loss_fn( self.model(), data).detach().numpy() print('Final loss: ', loss) # loss=closure().numpy() rec = self.model.rec.detach().numpy() return rec, loss
Note that “self.model” has trainable parameters that will be updated during LBFGS optimization. However, I get the following error in autograd. Weirdly, I don’t get the same error when using Adam. I know I am missing all the files in the error trace, but let me know if I can describe my issue in more detail.
File “/Users/mohan3/Desktop/Devs/Phase_Img/Code/phasetorch/phasetorch/nlpret.py”, line 111, in nlopt_phaseret
File “/Users/mohan3/Desktop/Devs/Phase_Img/Code/phasetorch/phasetorch/pret.py”, line 48, in run
File “/Users/mohan3/anaconda3/envs/phasetorch/lib/python3.8/site-packages/torch/autograd/grad_mode.py”, line 15, in decorate_context
return func(*args, **kwargs)
File “/Users/mohan3/anaconda3/envs/phasetorch/lib/python3.8/site-packages/torch/optim/lbfgs.py”, line 316, in step
flat_grad = self._gather_flat_grad()
File “/Users/mohan3/anaconda3/envs/phasetorch/lib/python3.8/site-packages/torch/optim/lbfgs.py”, line 255, in _gather_flat_grad
view = p.grad.view(-1)
RuntimeError: view size is not compatible with input tensor’s size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(…) instead.
Thank you very much for your time and help!