Hi all. For my project, I want to use the torch.optim.lr_scheduler.ReduceLROnPlateau learning rate scheduler to adjust my learning rate based on my validation loss. Looking at the documentation, however, it is not very clear to me what the threshold_mode parameter does, and which version of it, rel or abs is best to use?
The rel parameter is the default one, however, doing some quick calculations with pen and paper it appears that this version might lead to some very small margins when comparing validation losses between two iterations, much smaller than the threshold defined by me for measuring the new optimum.
Can anybody please advise on this? Cheers!