It seems like LBFGS optimizer cannot be copied with copy.deepcopy(). Other optimization method works fine.
The error prints:
\torch\optim\lbfgs.py", line 106, in step
flat_grad = self._gather_flat_grad()
\torch\optim\lbfgs.py", line 59, in _gather_flat_grad
for p in self._params:
AttributeError: ‘LBFGS’ object has no attribute ‘_params’