Hi, as far i know, SGD is doing:
x_new = x * learning_rate -gradient
When we take look at Adam what is Adam doing with gradient and learning rate ?
The docs show the applied formula and the source code might also be helpful to check how it’s implemented.
1 Like