How to set a fixed steps after which weights are updated, expect updating after every batch

[ptrblck]

The code looks alright and you don’t necessarily need to accumulate the loss, since the gradients will be automatically accumulated.
Here is a detailed description for different approaches.