Does "parameter.grad" is the gradient*lr or just gradient

[Btorch]

Hi, I want to measure the gradient norm, I have a working code, I just want to interpret it :

        for m,p in enumerate(model.parameters()):
            param_norm = p.grad.data.norm(2)
            total_norm += param_norm.item()

Now I wonder does p.grad is the already multiplied by the learning rate or the pure gradient?

Thanks a lot!

[sio277 (shoh)]

@Btorch

Now I wonder does p.grad is the already multiplied by the learning rate or the pure gradient?

No, it is just a gradient itself .