I have a question on ensemble model.
When I train an ensemble model, Should the base models be used in the state of finishing the training of each of them, or are they trained when an ensemble model is trained ?
How about the hyper-parameters of each base models and the ensemble model? Can they be different ?
These slides discuss different ensemble methods and might be helpful.
I will train say 8 models independently. So you end up with 8 different models were each model alone would give its own prediction. I then take take the average of the 8 predictions and that gives me my final prediction. In this case the ensemble model is just the average of the base models and thus doesn’t have any hyper parameters.
final_prediction = (Net1_pred+Net2_pred+….+Net7_pred+Net8_pred)/8
You could also take each models predictions and use them as features into another model that would give you the final prediction. In this case your final prediction model will have its own set of hyper parameters completely separate from you 8 base models.