Pre-calculating rnn input size to avoid hardcoding

Yes, reinitializing the module is the wrong approach and you should register it only once. Take a look at this post as well as the linked docs to check how to use the lazy module approach properly.