RNN implementation incorrect in PyTorch Tutorials?

1

38%20PM
38%20PM1164×728 68.8 KB

The RNN implementation in NLP From Scratch: Classifying Names with a Character-Level RNN — PyTorch Tutorials 2.2.0+cu121 documentation does not pass the hidden into the output but computes the output entirely separately. Isn’t this the equation of RNNs: 42%20PM
As described in http://blog.echen.me ?

2

Agree. When you check out simple RNN code result, it makes us wonder why the code in the RNN tutorial is different from the definition of RNN.

import torch.nn as nn
rnn = nn.RNN(10, 10, batch_first=True)

inp = torch.randn((1,3,10))

result = rnn(inp)

print(result[0].shape)
print(result[1].shape)
print(result[0][0])
print(result[1])

Same result of :
tensor([[[-0.0253, 0.5289, -0.2582, -0.5125, -0.4577, 0.5456, -0.4042,
-0.6476, -0.4210, -0.5473]]], grad_fn=)