Where is momentum in Adam method in the PYTORCH?

1

In the CAFFE, it has

Screenshot%20from%202018-06-12%2010-39-46
Screenshot%20from%202018-06-12%2010-39-461167×578 62.6 KB

We can find the setting in caffe/examples/mnist/lenet_solver_adam.prototxt at master · BVLC/caffe · GitHub

While in pytorch, the Adam method is

class torch.optim.Adam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False)

I did not find any params for momentum. How to set it in pytorch? Thanks

1 Like
2

Sorry. The last sentence mentioned it. beta is same as momentum in CAFFE

3

hi

  • lr → alpha
  • betas → (beta1, beta2)
4

Are you sure? So what is means “Caffes uses the value of momentum, momentum2 and delta for beta1,beta2 …” If I want to set momentum as 0.95, what should I do in PYTORCH with Adam method?

5

Any solution here? Is there a momentum equivalent for Adam?