Unfortunately, you can’t. You have to implement it your self as the layer norm are usually applied before the activation of the gates.
Fortunately, pytorch offers an Touchscript optimized implementation on Github. You can find it here.
Unfortunately, you can’t. You have to implement it your self as the layer norm are usually applied before the activation of the gates.
Fortunately, pytorch offers an Touchscript optimized implementation on Github. You can find it here.