Custom RNN Implementation

1

Hey,
I am interested to implement my own custom (Vanilla) RNN and I read nn.RNN but I am wondering that where is the mathematical operations (… math: h’ = \tanh(w_{ih} * x + b_{ih} + w_{hh} * h + b_{hh}) ) are taking place ? I want to modify these equations.
Would some one guide me in which file, I have to change them.

Thanks

1 Like
2

if you want to implement a custom RNN, it is better to do it by yourself in a separate file, rather than modifying the internals. Here’s an example of people implementing custom LSTMs in a separate file: Implementation of Multiplicative LSTM

1 Like
3

That makes sense. I’m trying to create a custom LSTM cell, but I’m struggling to get template LSTM code I got from the forums here to work. Would you mind taking a quick look? Thanks! Size mismatch error when using custom LSTM cell

1 Like