Issues with Chatbot NLP tutorial, Average loss not come down and generated sentence with meaningless ending

nlp
1

Hello, I posted an issue on offilce tutorial github.
Pls check https://github.com/pytorch/tutorials/issues/1029

First of all I run Cornell Movie-Dialogs Corpus, it performs good.

Then I changed the corpus to qingyun https://github.com/Doragd/Chinese-Chatbot-PyTorch-Implementation/blob/master/clean_chat_corpus/qingyun.tsv.
It works well too.

At last, I changed the corpus to our own chat corpus, this time it failed, when training peroid, the average loss stop coming down when reach to a high level, around 5%(when I use qingyun corpus it reached 0.32%). Besides the bot’s answer is obvious not correct, not related at all.

Do any one can help? Any parameters I can change, or our own corpus can not be used at all.
Thanks in advance!

2

While I understand you might get frustrated seeing your use case fail, I would like to ask you to use the proper language in this board, please.

I would recommend to scale down the use case and try to overfit a small data samples (e.g. just 10 samples) by playing around with the hyper-parameters of your training (learning rate, weigth decay etc.).
Once your model can properly overfit the sample, you could try to carefully scale up the use case again by adding more data and observing the training.

3

Thanks for your reply, sorry for my un-proper words.