Hey there pyTorch Community,
I am currently working with attentional neural networks for time series prediction in pyTorch.
During this I realized, that it would be great to have a norm argument for RNNCells which enables LayerNormalisation for them. I implemented it for the LSTMCell and it should be straight forward to apply to the other ones as well.
I think it would be worthwhile to implement it and I would be happy to do so, as it allows first time users to utilize a layer norm without going down to torch/nn/functions_/rnn.
What do you think?
Cheers,
Jendrik