# Averaged SGD implementation

**URL:** <https://discuss.pytorch.org/t/averaged-sgd-implementation/26960>\
**Category:** autograd\
**Created:** [October 10, 2018, 12:03pm UTC](https://discuss.pytorch.org/t/averaged-sgd-implementation/26960 "2018-10-10T12:03:10Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ramsama1624](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/r/6a8cbe/32.png) [@ramsama1624](https://discuss.pytorch.org/u/ramsama1624)\
**Post date:** [October 10, 2018, 12:03pm UTC](https://discuss.pytorch.org/t/averaged-sgd-implementation/26960/1 "2018-10-10T12:03:10Z")

</div>

Hi, I’m looking into ASGD implementation and [here](https://github.com/pytorch/pytorch/blob/727609f435582f70a57775f04694f619ff8add80/torch/optim/asgd.py#L75), it seems like averaged weight is saved in state[‘ax’]. As far as I see it, there should be a line of code copying state[‘ax’] to p.data. However I don’t see anywhere state[‘ax’] is used for updating parameters. Am I missing something?

---

<div class="post-metadata">

**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\
**Post date:** [October 10, 2018, 1:46pm UTC](https://discuss.pytorch.org/t/averaged-sgd-implementation/26960/2 "2018-10-10T13:46:41Z")

</div>

I don’t think you’re supposed to use the average as parameters.  
The average is just the final solution.  
Or is it me that misread the paper?
