In the tutorial http://pytorch.org/tutorials/intermediate/char_rnn_generation_tutorial.html. Why the same input could produces different output? After training, the model parameters supposed to be fixed, isn’t it?
It looks like you solved it already but the reason this happens is the dropout layer adding some randomness. The dropout could be “turned off” to make the model deterministic with rnn.train(False) in the sample function.
Hi, spro, I’m a begginner of DL and Pytorch, I cannt understand why the net graph in http://pytorch.org/tutorials/intermediate/char_rnn_generation_tutorial.html is that, I donnt think the previous output is the next step input, could you tell me why? Thanks!
When generating, the previous output is the next input, however when training, the correct output is the next input. This training technique is known as “teacher forcing” - look that up for more on why it’s used.
1 Like
Thanks very much for your explanation!