How to clip the gradient?

How to clip the gradient in a model during training alone ? Please help

Gradient clipping is always only done in training (because you ordinarily don’t do backprop in evaluation).
There are two ways:

  • backpropagate, then clip gradients using the utility functions,
  • add backward hooks doing the clipping in flight.

You might see this somewhat old thread for a discussion of the details:

Best regards

Thomas