# Grad sharing problem in a3c

**URL:** <https://discuss.pytorch.org/t/grad-sharing-problem-in-a3c/10635>\
**Category:** autograd\
**Created:** [December 1, 2017, 1:23am UTC](https://discuss.pytorch.org/t/grad-sharing-problem-in-a3c/10635 "2017-12-01T01:23:04Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vincent\_Lee](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vincent_lee/32/2095_2.png) [@Vincent\_Lee](https://discuss.pytorch.org/u/Vincent_Lee)\
**Post date:** [December 1, 2017, 1:23am UTC](https://discuss.pytorch.org/t/grad-sharing-problem-in-a3c/10635/1 "2017-12-01T01:23:04Z")

</div>

> <https://github.com/ikostrikov/pytorch-a3c/pull/9/files>

In ensure\_shared\_grads, wouldn’t shared\_param.grad NOT be None after the first time the parameters are updated? In my testing it seems that the function returns without syncing the gradient in every step after the first. Related discussion here.

> [@Problem on Variable.grad.data?](https://discuss.pytorch.org/t/problem-on-variable-grad-data/957/16):
>
> Thank you~ For the safe issue, i noticed that DM’s paper explicitly said they don’t put a lock on the shared weights. For your solution to the shared grad to pytorch-a3c. def ensure\_shared\_grads(model, shared\_model): for param, shared\_param in zip(model.parameters(), shared\_model.parameters()): if shared\_param.grad is not None: return shared\_param.\_grad = param.grad Will this code restrain the shard\_model grad only being bounded with one local\_model? Cause sha…

```auto
def ensure_shared_grads(model, shared_model):
    for param, shared_param in zip(model.parameters(), shared_model.parameters()):
        if shared_param.grad is not None:
            return
        shared_param._grad = param.grad

```

In DM’s paper they say they perform async updates without locks. What does it mean?

what will happen if `shared_param.grad` is changed by another thread after returning from `ensure_shared_grads`?  
Will the gradient of the current thread be discarded?

---

<div class="post-metadata">

**Author:** ![dgriff](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dgriff/32/1361_2.png) [@dgriff](https://discuss.pytorch.org/u/dgriff)\
**Post date:** [December 11, 2017, 5:55am UTC](https://discuss.pytorch.org/t/grad-sharing-problem-in-a3c/10635/2 "2017-12-11T05:55:38Z")

</div>

yeah it really there just to make sure initializing goes smoothly.

async updates without locks mean they updates from the parallel processes are done asynchronously which means they are run on their own clock and the different process threads can actually make updates at the same time which can lead to bad updates. The reason they do this even though there is risk of bad updates being made is that updates are done much faster which speeds up training as the process of acquiring and releasing locks to do synchronous updates very noticeably slows down the update speed. The positives of much more frequent updates hopefully outweigh the possible negatives of bad updates made and overall speed up training of model