Hi,
I have a simple rnn code below.
rnn = nn.RNN(1, 1, 1, bias = False, batch_first = True)
t = torch.ones(size = (1, 2, 1))
output, hidden = rnn(t)
My understanding from the documentation is that the output from above is the hidden state.
I tried to manually calculate the output using the below
hidden_state1 = torch.tanh(t[0][0] * rnn.weight_ih_l0)
hidden_state2 = torch.tanh(t[0][1] * rnn.weight_ih_l0 + hidden_state1 * rnn.weight_hh_l0)
The result was correct. hidden_state1 and hidden_state2 match the output.
Shouldn’t the hidden_states get multiplied with output weights to get the output?
I checked for weights connecting from hidden state to output. But there are no weights at all.
Is there anything I’m missing? Please help me.