How to do exponential learning rate decay in PyTorch?

autograd
1

Hi! I want to transform the codes below implemented with TensorFlow into a PyTorch version:

lr = tf.train.exponential_decay(start_lr, global_step, 3000, 0.96, staircase=True)
optimizer = tf.train.AdamOptimizer(learning_rate=lr, epsilon=0.1)

But I don’t know what’s the counterpart of PyTorch of exponential learning rate decay. Anyone who can tell me? Thanks a lot!

5 Likes