Momentum in SGD

Soumava_Roy (Soumava Roy) August 6, 2018, 1:10am 1

It seems that the final value of momentum is (learning_rate * momentum) in SGD; which is not according to the standard SGD equations.

SimonW (Simon Wang) August 6, 2018, 4:09am 2

see here for details: https://github.com/pytorch/pytorch/issues/1099