Gradient clipping

[Brando_Miranda (MirandaAgent)]

its really annoying cuz clamp_ is not documented. I guess one just has to randomly try functions with an underscore at the end and see if they exist. Thanks though.

---

similar Q:

Proper way to do gradient clipping?

Is there a proper way to do gradient clipping, for example, with Adam? It seems like that the value of Variable.data.grad should be manipulated (clipped) before calling optimizer.step() method. I think the value of Variable.data.grad can be modified in-place to do gradient clipping. Is it safe to do? Also, Is there a reason that Autograd RNN cells have separated biases for input-to-hidden and hidden-to-hidden? I think this is redundant and has a some overhead.