Adding a layer to an RNN

1

Hi there,

So I’ve been trying to make a certain kind of Neural Network to work. My layout is the following: I want my input first to go into a hidden layer of x neurons then it goes through another layer where it gives the output (so a two layer network). Now the “tricky” (for a newly introduced to PyTorch as myself) part is that I want to give the output at time t as a new feature in the input, so that my input constitutes of x(t),y(t-1). My instinct was that it is some kind of Recurrent Neural Net but as I tried to implement that I have no possibility of adding an x-sized layer into the RNN. Am I wrong ? As I see it the RNN has only one layer of which you cannot determine the hidden layer size. How should I proceed ?

Thank you very much if you have the time to help :smiley:

2

To use multiple inputs, you can concatenate them on last dimension (torch.cat).

Note that you may not need x/y role split in the middle of the network, unless you have residual architecture, multiple outputs/losses or something like that - your second hidden layer can produce hidden x representation that is fed to rnn, and Y can be created later.