I am implementing the review net (link: https://github.com/kimiyoung/review_net/blob/master/image_caption_offline/reason_att_copy.lua ). It is just an ordinary encoder-decoder framework plus review steps. The review step is similar to decoder step, but there is no input and the weights of LSTMs are not shared. In my implementation, it seems that the parameters of LSTMs in review steps are not updated when training. If the review steps are deleted, the code works fine and I can get the correct results.