# Optimizer problem when copying a model

**URL:** <https://discuss.pytorch.org/t/optimizer-problem-when-copying-a-model/158319>\
**Category:** Uncategorized\
**Created:** [August 4, 2022, 8:31am UTC](https://discuss.pytorch.org/t/optimizer-problem-when-copying-a-model/158319 "2022-08-04T08:31:29Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![kharrat\_salma](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kharrat_salma/32/46473_2.png) [@kharrat\_salma](https://discuss.pytorch.org/u/kharrat_salma)\
**Post date:** [August 4, 2022, 8:31am UTC](https://discuss.pytorch.org/t/optimizer-problem-when-copying-a-model/158319/1 "2022-08-04T08:31:29Z")

</div>

Hello, I have to train _n_ clients where each client has its own model

so I instantiated a model `model=Net()` and then created _n_ copies:

`models.append(copy.deepcopy(model))`

after every batch I aggregate all weights and do the following:  
`models[i].load_state_dict(average_weights)`

I noticed that there is no learning, and the loss gets stuck.

 ![image](https://discuss.pytorch.org/uploads/default/original/3X/f/5/f56a91f97ddab75e6f01105ea6167fe713660085.png)

Any suggestion for this problem?

---

<div class="post-metadata">

**Author:** ![sudomaze](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sudomaze/32/31865_2.png) [@sudomaze](https://discuss.pytorch.org/u/sudomaze)\
**Post date:** [August 4, 2022, 11:48am UTC](https://discuss.pytorch.org/t/optimizer-problem-when-copying-a-model/158319/2 "2022-08-04T11:48:12Z")

</div>

Why do you have multiple models and then average their weights?

Each model might learn something different, and taking the average of the weights, especially during training, won’t result in a better estimation to solve the problem at hand.