This is taken from here GitHub - davda54/sam: SAM: Sharpness-Aware Minimization (PyTorch)

You are right, I fixed the first one but the second method is still giving an error

```
#---------------Defination of LLOSS------------
if method == 'lloss':
base_optimizer = torch.optim.SGD
optim_module = SAM(models['module'].parameters(), base_optimizer, lr=LR,
momentum=MOMENTUM, weight_decay=WDECAY)
sched_module = lr_scheduler.MultiStepLR(optim_module, milestones=MILESTONES)
optimizers = {'backbone': optim_backbone, 'module': optim_module}
schedulers = {'backbone': sched_backbone, 'module': sched_module}
# -----------------SAM Optimizer -------------------
criterion(models['backbone'](inputs)[0], labels)
loss.backward(retain_graph=True)
optimizers['backbone'].first_step(zero_grad=True)
criterion(models['backbone'](inputs)[0], labels)
optimizers['backbone'].second_step(zero_grad=True)
# -----------------SAM Optimizer for LLOSS Method -------------------
if method == 'lloss':
#optimizers['module'].step()
criterion(models['backbone'](inputs)[0], labels)
loss.backward(retain_graph=True)
optimizers['module'].first_step(zero_grad=True)
criterion(models['backbone'](inputs)[0], labels)
optimizers['module'].second_step(zero_grad=True)
```

**ERROR**

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [512, 100]], which is output 0 of AsStridedBackward0, is at version 3; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).