# How to set a fixed steps after which weights are updated, expect updating after every batch

**URL:** <https://discuss.pytorch.org/t/how-to-set-a-fixed-steps-after-which-weights-are-updated-expect-updating-after-every-batch/75251>\
**Category:** Uncategorized\
**Created:** [April 3, 2020, 4:10pm UTC](https://discuss.pytorch.org/t/how-to-set-a-fixed-steps-after-which-weights-are-updated-expect-updating-after-every-batch/75251 "2020-04-03T16:10:43Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![prateekgupta891](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/p/ecc23a/32.png) [@prateekgupta891](https://discuss.pytorch.org/u/prateekgupta891)\
**Post date:** [April 3, 2020, 4:10pm UTC](https://discuss.pytorch.org/t/how-to-set-a-fixed-steps-after-which-weights-are-updated-expect-updating-after-every-batch/75251/1 "2020-04-03T16:10:43Z")

</div>

Image Size varies a lot. Hence kept the Batch size = 1.  
Now i need to propogate back the loss and do the optimizer step after every BACK\_STEP no. of batches ( batch\_size = 1).

```auto
model.train()
BACK_STEP = 128
for batch_idx, (inputs,labels) in enumerate(train_loader):
    inputs = inputs.to(device)
    labels = labels.to(device)

    if (batch_idx)%BACK_STEP==0:
        optimizer.zero_grad()

    with torch.set_grad_enabled(True):
        outputs = model(inputs)
        _, preds = torch.max(outputs, 1)
        loss = criterion(outputs, labels)
        #losses +=loss
        #print(labels,outputs,preds,loss)
        loss.backward()
        if (batch_idx+1)%BACK_STEP==0:
            optimizer.step()

    # statistics
    running_loss += loss.item() * inputs.size(0)
    running_corrects += torch.sum(preds == labels.data)
    
scheduler.step()

```

Is it the correct way to do it?  
Because I think loss needs to be added as well for all images passes and then propogated back.  
Any changes you would suggest?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [April 4, 2020, 7:19am UTC](https://discuss.pytorch.org/t/how-to-set-a-fixed-steps-after-which-weights-are-updated-expect-updating-after-every-batch/75251/2 "2020-04-04T07:19:51Z")

</div>

The code looks alright and you don’t necessarily need to accumulate the loss, since the gradients will be automatically accumulated.  
[Here](https://discuss.pytorch.org/t/why-do-we-need-to-set-the-gradients-manually-to-zero-in-pytorch/4903/20) is a detailed description for different approaches.