What is the title type of this ensemble?

1

i used those functions to merge between two models but i don’t know the type of this ensemble

class MyEnsemble(nn.Module):
    def __init__(self, modelA, modelB):
        super(MyEnsemble, self).__init__()
        self.modelA = modelA
        self.modelB = modelB
    def forward(self, x):
        x1 = self.modelA(x.clone()) 
        x1 = x1.view(x1.size(0), -1)
        x2 = self.modelB(x)
        x2 = x2.view(x2.size(0), -1)
        x = torch.cat((x1, x2), dim=1)

will that be Bootstrap Aggregation Ensemble or K-fold Cross-Validation Ensemble or Boosting Ensemble?

2

I don’t think any of the mentioned techniques are seen in the code example.
If I’m not mistaken the mentioned techniques are:

  • Bootstrap Aggregation Ensemble (bagging): would train weak learners on subsets of the training data and would then create an ensemble for the final prediction
  • K-fold Cross-Validation Ensemble: I’m not familiar with this ensemble technique, but guess that you would be using k-fold CV to create weak learners again and put them into an ensemble. Note that k-fold CV is usually applied for hyperparameter search.
  • Boosting Ensemble: would sequentially create weak learners by feeding them with misclassified samples from the previous stage (or by applying a larger weight for misclassified samples).

In your example you are only concatenating the outputs of two different models, so I also wouldn’t call it an ensemble without getting a final prediction.

2 Likes
3

thanks for replying but excuse me what do you mean by predictions … as I searched for how to ensemble two different models and found this way by concatenating the outputs of the two models then taking this merge and training the model with it … if it is not ensemble what is this technique called, please

4

Yes, this would be a 2-stage ensemble where the outputs of stage0 are fed to a classifier in stage1 (it could be any trained classifier, a voting classifier etc.).
In your current code snippet you are just concatenating the outputs of two base models without adding any classifier on top, so I wouldn’t call it an ensemble.

1 Like
6

I don’t want to nitpick, but I wouldn’t call it an ensemble model, since you are unable to get predictions using your current code snippet and are just concatenating features or different models.
Depending on the training strategy and the classification process you are using, it could be used in an ensemble. Feature concatenation itself is also used in e.g. UNets and I wouldn’t call the internal UNet architecture an ensemble of models.

1 Like
9

excuse me and thanks for your time as I’m conflicting in what I did. if you can help me if this way called merging model, not ensemble model as I searched but need to confirm … the basic idea is to get a model from two models to use it in finding the features of the images using PyTorch so I thought that deleting the classifier from both then merge is the right way … is there any way to use ensemble and what I did in the code called merging not ensemble?

appreciate your reply