oh, You told to average weights and put on model3, I’ve misunderstood it,
I think your question is really to ensemble two or more pretrained model, here you can do that
https://discuss.pytorch.org/t/combining-trained-models-in-pytorch/28383
Hope it solves your problem