Yes I remember this solution from here. I think it’s good in the case of two models, but I don’t think it’s the most efficient in the case of a sequence of many models. Because as you see, both in the code you suggested and in the one I pasted with this question, we are having a double loop, so this is very slow, because of the fact that loops in python are slow.
Normally, having some option that enables averaging models, can be very useful, for example for Stochastic Weighted Averaging or other variations of this method, and I would be surprised if pytorch doesn’t support this feature.