Question about model ensembling performance

Hi,

Suppose I want to ensemble two models, I find two method, one is like this:

model1 = torchvision.models.resnet50().cuda()
model2 = torchvision.models.resnet101().cuda()

inten = torch.randn(16, 3, 224, 224).cuda()
n_iter = 1000

for i in range(n_iter):
    out = model1(inten).softmax(1)
    out += model2(inten).softmax(1)

And another method is to define these two models as children of a nn.Module:

class Model(torch.nn.Module):

    def __init__(self):
        super().__init__()
        self.model1 = torchvision.models.resnet50().cuda()
        self.model2 = torchvision.models.resnet101().cuda()

    def forward(self, x):
        out = self.model1(x).softmax(1)
        out += self.model2(x).softmax(1)
        return out

model = Model().cuda()

for i in range(n_iter):
    out = model(inten)

Which one is faster and better optimized during inference ? Will there be difference in terms of speed and memory usage between these two methods ?

Both methods should yield the same performance.
The two code snippets would take slightly different Python paths, but the overhead shouldn’t be visible in your profiling.

1 Like