Step Activation Function

[KFrank]

Hello Ziqi and Ayman!

@ziqipang

However, I think your function is not differentiable, so you might have to be careful on using this function.

tanh() is a commonly-used differentiable approximation to the
step function, and is sometimes used as an activation function.
(We often call these differentiable approximations “soft” versions
of the functions they approximate.)

Best.

K. Frank