Non scalar backward and self mini batch implementation

[meipuru344]

I appreciate your reply

In this case

torch.autograd.backward([output],[grad])

Is grad of each data in minibatch accumulated like bellow?

How to implement accumulated gradient in pytorch (i.e. iter_size in caffe prototxt)

how to can i accumulate gradient during gradient descent in pytorch (i.e. iter_size in caffe prototxt). Currently, my code is: for iter, (images, labels, indices) in enumerate(train_loader, 0): optimizer.zero_grad() outputs = net(Variable(images.cuda())) loss = criterion(outputs, Variable(labels.cuda())) loss.backward() optimizer.step() Do i do this? for iter in range(N): optimizer.zero_grad() …

Thanks