Mini-batch gradient or single gradient?

Hi All, I have a question about the backward. In the following backward method, grad_output is the gradient with respect to the output. My question is that grad_output is computed from a single sample or mini-batch samples? Thanks in advance! @smth

Inherit from Function

class LinearFunction(Function):

# Note that both forward and backward are @staticmethods
# bias is an optional argument
def forward(ctx, input, weight, bias=None):
    ctx.save_for_backward(input, weight, bias)
    output =
    if bias is not None:
        output += bias.unsqueeze(0).expand_as(output)
    return output

# This function has only a single output, so it gets only one gradient
def backward(ctx, grad_output):
    # This is a pattern that is very convenient - at the top of backward
    # unpack saved_tensors and initialize all gradients w.r.t. inputs to
    # None. Thanks to the fact that additional trailing Nones are
    # ignored, the return statement is simple even when the function has
    # optional inputs.
    input, weight, bias = ctx.saved_tensors
    grad_input = grad_weight = grad_bias = None

    # These needs_input_grad checks are optional and there only to
    # improve efficiency. If you want to make your code simpler, you can
    # skip them. Returning gradients for inputs that don't require it is
    # not an error.
    if ctx.needs_input_grad[0]:
        grad_input =
    if ctx.needs_input_grad[1]:
        grad_weight = grad_output.t().mm(input)
    if bias is not None and ctx.needs_input_grad[2]:
        grad_bias = grad_output.sum(0).squeeze(0)

    return grad_input, grad_weight, grad_bias