How to make the parameter of torch.nn.Threshold learnable?

21

I’ve been experimenting with learning the threshold parameters for expressions like .clamp(min=lower) where ‘lower’ is a Module Parameter.

Here is a function for accomplishing it for clamping to zero or negative values:

def Clamp(x, minval):
    """
    Clamps Variable x to minval.
    minval <= 0.0
    """
    return x.clamp(max=0.0).sub(minval).clamp(min=0.0).add(minval) + x.clamp(min=0.0)

With some extra work the same could be done for .clamp(max=upper) .

1 Like