Non scalar backward and self mini batch implementation

[meipuru344]

Does PyTorch average or sum gradients over a minibatch?

For example when you retrieve the gradients like so: loss = F.nll_loss(output, target) loss.backward() for key, value in model.named_parameters(): mygrad = value.grad is mygrad the sum of gradients over the minibatch or the average?

My question relates this one.

#Quote Mr SimonW
All autograd does is just to calculate the gradient, it has no notion of batching, and I don’t see how it can have different behavior with different batching mechanism.