Proper way to do gradient clipping?

13

for people trying to just get an answer quickly:

torch.nn.utils.clip_grad_norm(mdl_sgd.parameters(),clip)

or with in-place clamp:

W.grad.data.clamp_(-clip,clip)

also similar Q:

40 Likes