Merging model after trainer, before evaluation and predict

nlp
1

Hi everyone,

I have two models of the Trainer class from Transformers lib.

In particular:

trainer1 = Trainer(
        model=model1,
        args=training_args,
        train_dataset=train_dataset,
        eval_dataset=eval_dataset,
        compute_metrics=compute_metrics,
    )

trainer2 = Trainer(
        model=model2,
        args=training_args,
        train_dataset=train_dataset,
        eval_dataset=eval_dataset,
        compute_metrics=compute_metrics,
    )

both models were trained (trainer.train(model_path=...) trainer.save_model()) on the same dataset to which I made some changes.

Now my problem is that I necessarily need to merge these two models (merge their weights (does that make sense?) or something like that) before doing the evaluation (trainer.evaluate()) and prediction (trainer.predict(test_dataset)) operations.

How can I do?
Do you have any suggestions?

P.s. The important thing is that the model resulting from the union/merge of these two models always has a Trainer type.

Thanks in advance!

2

Since this question is Huggingface-specific, I think you might get a good answer in their forum. :slight_smile: