 # How to assing average of two models' weights into a new model?

Hi, so i have this network:

``````class Network(nn.Module):
def __init__(self):
super(Network, self).__init__()

self.conv2d = nn.Sequential(
nn.Conv2d(1, 64, (3,6), (1, 1)),
nn.Hardsigmoid()
)

def forward(self, input):
output = self.conv2d(input)

return output
``````

The network is trained on two different dataset so to obtain two different models:

``````model1
model2
``````

my objective is to average the weights and bias of them and put the result on a third model:
`model3`

How can i achieve that?

1 Like

Cool question, I’ve tried,

I think, here’s you can solve this,
We can get weights of any model by `model.parameters()` which can be append into list as below

``````params1 = []
for param in model1.parameters():
params1.append(param.data)
``````

similary do this to trained model2 and save in list `params2`

Now you initialize weights of model3 as

``````model3 = Network()
params3 = iter(params1 + params2)
for param in model3.parameters():
param.data = next(params3)

``````

but in this way there is no chance to average the weight from model1 and model2 right?

oh, You told to average weights and put on model3, I’ve misunderstood it,

I think your question is really to ensemble two or more pretrained model, here you can do that
https://discuss.pytorch.org/t/combining-trained-models-in-pytorch/28383

no in the post that you sended me he combine two models, my goal is to get two identical model, get weights and bias and average them into a third model that has the same structure of the previous two

model parameters are actually weights and biases. As earlier, you can also get name of those parameters as

``````for name, params in model.named_parameters():
print(name)
``````

gives your weights and biases names as

``````conv2d.0.weight
conv2d.0.bias
dense.0.weight
dense.0.bias
``````