I try to build a transformer to generate text, my src_input is sentence, tgt_input is +sentence, tgt_output is sentence+, masks is the default setting. But when I training, the loss is very low (about 0.0001, I have trained same dataset using RNN, the loss is 0.1), this results in the generated text being almost unique (no matter how I adjust the temperature sampling value).
Is there something wrong with my masks, should I set my src_mask and memory_mask?
thanks！