How to implement a customize weight

[Eta_C]

Yes, if ONLY CLIP, check here

Restrict range of variable during gradient descent

For your example (constraining variables to be between 0 and 1), there’s no difference between what you’re suggesting – clipping the gradient update – versus letting that gradient update take place in full and then clipping the weights afterwards. Clipping the weights, however, is much easier than modifying the optimizer. Here’s a simple example of a UnitNorm clipper: class UnitNormClipper(object): def __init__(self, frequency=5): self.frequency = frequency def __call__(self,…