Calculating loss for entire batch using NLLLoss in 0.4.0

[Defake (Genych Defake)]

It doesn’t make sense. I need to accumulate gradient.
For example:

How to implement accumulated gradient in pytorch (i.e. iter_size in caffe prototxt)

how to can i accumulate gradient during gradient descent in pytorch (i.e. iter_size in caffe prototxt). Currently, my code is: for iter, (images, labels, indices) in enumerate(train_loader, 0): optimizer.zero_grad() outputs = net(Variable(images.cuda())) loss = criterion(outputs, Variable(labels.cuda())) loss.backward() optimizer.step() Do i do this? for iter in range(N): optimizer.zero_grad() …