diff b/w grad.zero_ or zero_grad

tejaswi_lakshmi · February 25, 2020, 2:26pm

tom · February 25, 2020, 2:41pm

Convenience and detach_ (in case you have used build_graph=True):

pytorch/pytorch/blob/4460c8b034f8fd544ff9c271c4aa21698644d352/torch/optim/optimizer.py#L159-L166


def zero_grad(self):
    r"""Clears the gradients of all optimized :class:`torch.Tensor` s."""
    for group in self.param_groups:
        for p in group['params']:
            if p.grad is not None:
                p.grad.detach_()
                p.grad.zero_()

Best regards

Thomas