Where is momentum in Adam method in the PYTORCH?

In the CAFFE, it has

We can find the setting in https://github.com/BVLC/caffe/blob/master/examples/mnist/lenet_solver_adam.prototxt#L12

While in pytorch, the Adam method is

class torch.optim.Adam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False)

I did not find any params for momentum. How to set it in pytorch? Thanks

1 Like

Sorry. The last sentence mentioned it. beta is same as momentum in CAFFE


  • lr β†’ alpha
  • betas β†’ (beta1, beta2)

Are you sure? So what is means β€œCaffes uses the value of momentum, momentum2 and delta for beta1,beta2 …” If I want to set momentum as 0.95, what should I do in PYTORCH with Adam method?

Any solution here? Is there a momentum equivalent for Adam?