# Where is momentum in Adam method in the PYTORCH?

**URL:** <https://discuss.pytorch.org/t/where-is-momentum-in-adam-method-in-the-pytorch/19536>\
**Category:** Uncategorized\
**Created:** [June 12, 2018, 1:41am UTC](https://discuss.pytorch.org/t/where-is-momentum-in-adam-method-in-the-pytorch/19536 "2018-06-12T01:41:53Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![John1231983](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/john1231983/32/4040_2.png) [@John1231983](https://discuss.pytorch.org/u/John1231983)\
**Post date:** [June 12, 2018, 1:41am UTC](https://discuss.pytorch.org/t/where-is-momentum-in-adam-method-in-the-pytorch/19536/1 "2018-06-12T01:41:53Z")

</div>

In the CAFFE, it has

 ![Screenshot%20from%202018-06-12%2010-39-46](https://discuss.pytorch.org/uploads/default/original/2X/d/d058975e25fdf004daa552195ad1cd1b0fc52548.png)

We can find the setting in [caffe/examples/mnist/lenet\_solver\_adam.prototxt at master · BVLC/caffe · GitHub](https://github.com/BVLC/caffe/blob/master/examples/mnist/lenet_solver_adam.prototxt#L12)

While in pytorch, the Adam method is

```auto
class torch.optim.Adam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False)

```

I did not find any params for momentum. How to set it in pytorch? Thanks

---

<div class="post-metadata">

**Author:** ![John1231983](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/john1231983/32/4040_2.png) [@John1231983](https://discuss.pytorch.org/u/John1231983)\
**Post date:** [June 12, 2018, 2:06am UTC](https://discuss.pytorch.org/t/where-is-momentum-in-adam-method-in-the-pytorch/19536/2 "2018-06-12T02:06:28Z")

</div>

Sorry. The last sentence mentioned it. beta is same as momentum in CAFFE

---

<div class="post-metadata">

**Author:** ![crcrpar](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/crcrpar/32/24766_2.png) [@crcrpar](https://discuss.pytorch.org/u/crcrpar)\
**Post date:** [June 12, 2018, 6:31am UTC](https://discuss.pytorch.org/t/where-is-momentum-in-adam-method-in-the-pytorch/19536/3 "2018-06-12T06:31:36Z")

</div>

hi

- `lr` → `alpha`
- `betas` → (beta1, beta2)

---

<div class="post-metadata">

**Author:** ![John1231983](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/john1231983/32/4040_2.png) [@John1231983](https://discuss.pytorch.org/u/John1231983)\
**Post date:** [June 12, 2018, 7:50am UTC](https://discuss.pytorch.org/t/where-is-momentum-in-adam-method-in-the-pytorch/19536/4 "2018-06-12T07:50:32Z")

</div>

Are you sure? So what is means “Caffes uses the value of momentum, momentum2 and delta for beta1,beta2 …” If I want to set momentum as 0.95, what should I do in PYTORCH with Adam method?

---

<div class="post-metadata">

**Author:** ![krioux5](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/848f3c/32.png) [@krioux5](https://discuss.pytorch.org/u/krioux5)\
**Post date:** [July 30, 2020, 12:35pm UTC](https://discuss.pytorch.org/t/where-is-momentum-in-adam-method-in-the-pytorch/19536/5 "2020-07-30T12:35:01Z")

</div>

> [@crcrpar](#):
>
> → `alpha`

Any solution here? Is there a momentum equivalent for Adam?