Custom RNNs: what is the best strategy to implement them?

nlp
1

Hi all,

I’d like to implement a custom RNN architecture and I’m trying to understand which is the best strategy, from the computational point of view, to do it.

After some search in this (fantastic) forum, I realized there are several ways to proceed:

  1. Use a for loop inside a forward to compute the dynamic evolution of the network
  2. Manually unroll the RNN (the length of the input is fixed), which would be similar to what done in TF v1 by the function dynamic_rnn
  3. Follow this PyTorch post which however looks a bit more complicated, at least for me, as I am a beginner with PyTorch.

Thank you for your precious advices!