Keras provides two functions which are fairly straightforward to implement, and everyone loves them:
This one reduces LR when gradient is stuck on a plateau for past “X=patience” epochs:
ReduceLROnPlateau(monitor='loss_value', factor=np.sqrt(0.1), cooldown=0, patience=10, min_lr=0.5e-6, verbose=1)
This one stops you from burning up your Amazon AWS $$$ credits if your model is not learning anything after “X=patience” epochs:
EarlyStopping(monitor='accuracy_value', min_delta=0.0001, patience=15, verbose=1)
Would PyTorch be open to adding something like this?