Hi guys,
I have a RNN with tabular data that needs to predict over 30 outputs at each time step. What I have observed is that the prediction of one output damages the other. Can you guys recommend some literature on how to specialize your NN to predict different outputs? I know that, for autonomous driving, you have many outputs being generated at the same time.
Thanks in advance!
What I have observed is that the prediction of one output damages the other?
Iām assuming your problem is a multi task, multi class problem. This is the essence of Multi Task learning in general. You utilize inductive bias when training on mulitple tasks, thus allowing your model to generalize better to all the tasks. The roots of this idea trace way back to 1993: Multitask Learning | Machine Learning. I am not an expert in MTL for NLP. However, I just went through this blog: https://ruder.io./multi-task-learning-nlp/ . This is a good primer for MTL in NLP. Do go through the citations to get an idea of the basics of MTL fot NLP. In general, the relatability of tasks seems to be very important to achieve generalization. I would suggest going through the latest work in ACL, EMNLP or NIPS to get a better idea about the cutting edge in MTL for NLP. Do let me know if you come across a good paper yourself.