# Get current LR of optimizer with adaptive LR **URL:** \ **Category:** Uncategorized\ **Created:** [September 10, 2018, 8:23pm UTC](https://discuss.pytorch.org/t/get-current-lr-of-optimizer-with-adaptive-lr/24851 "2018-09-10T20:23:32Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![marcman411](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/7cd45c/32.png) [@marcman411](https://discuss.pytorch.org/u/marcman411)\ **Post date:** [September 10, 2018, 8:23pm UTC](https://discuss.pytorch.org/t/get-current-lr-of-optimizer-with-adaptive-lr/24851/1 "2018-09-10T20:23:33Z")
How can I get the current learning rate being used by my optimizer? Many of the optimizers in the [`torch.optim`](https://pytorch.org/docs/stable/optim.html#algorithms) class use variable learning rates. You can provide an initial one, but they should change depending on the data. I would like to be able to check the current rate being used at any given time. This question is basically a duplicate of [this one](https://discuss.pytorch.org/t/print-current-learning-rate-of-the-adam-optimizer/15204), but I don’t think that one was very satisfactorily answered. Using Adam, for example, when I print: ```auto optimizer = torch.optim.Adam(model.parameters(), lr=1e-4) '''...training...''' for param_group in optimizer.param_groups: print(param_group['lr']) ``` I always see the initial learning rate no matter how many epochs of data I run. However, if I want to restart interrupted training progress or even just debug my loss, it makes sense to know where the optimizer left off. How can I do this? ---
**Author:** ![TheShadow29](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/theshadow29/32/2013_2.png) [@TheShadow29](https://discuss.pytorch.org/u/TheShadow29)\ **Post date:** [September 10, 2018, 8:44pm UTC](https://discuss.pytorch.org/t/get-current-lr-of-optimizer-with-adaptive-lr/24851/2 "2018-09-10T20:44:08Z")
You could save the optimizer state dict. Something like this suggested here: [https://github.com/pytorch/pytorch/issues/2830#issuecomment-336194949](https://github.com/pytorch/pytorch/issues/2830#issuecomment-336194949) should work. After that you could just load it back. I am guessing the reason you see the same lr is that the lr for adam has not changed. The effective lr which has components from moment estimate are different. ---
**Author:** ![marcman411](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/7cd45c/32.png) [@marcman411](https://discuss.pytorch.org/u/marcman411)\ **Post date:** [September 10, 2018, 8:51pm UTC](https://discuss.pytorch.org/t/get-current-lr-of-optimizer-with-adaptive-lr/24851/3 "2018-09-10T20:51:40Z")
That makes sense; thanks for the link. Is there a way to show the effective learning rate, short of calculating it on my own? ---
**Author:** ![TheShadow29](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/theshadow29/32/2013_2.png) [@TheShadow29](https://discuss.pytorch.org/u/TheShadow29)\ **Post date:** [September 10, 2018, 8:56pm UTC](https://discuss.pytorch.org/t/get-current-lr-of-optimizer-with-adaptive-lr/24851/4 "2018-09-10T20:56:07Z")
I am sorry, but I am unaware of any direct way. ---
**Author:** ![kengz](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kengz/32/25177_2.png) [@kengz](https://discuss.pytorch.org/u/kengz)\ **Post date:** [May 30, 2020, 10:06pm UTC](https://discuss.pytorch.org/t/get-current-lr-of-optimizer-with-adaptive-lr/24851/5 "2020-05-30T22:06:53Z")
for most optim all layers use the same lr, so u can just do: ```auto print(optimizer.param_groups[0]['lr']) ``` If you’re using a lr\_scheduler u can do the same, or use: ```auto print(lr_scheduler.get_lr()) ``` ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [May 31, 2020, 10:16am UTC](https://discuss.pytorch.org/t/get-current-lr-of-optimizer-with-adaptive-lr/24851/6 "2020-05-31T10:16:30Z")
Nit: `get_lr()` might not yield the current learning rate, so you should use `get_last_lr()`. 😉 ---
**Author:** ![dinarkino](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dinarkino/32/23356_2.png) [@dinarkino](https://discuss.pytorch.org/u/dinarkino)\ **Post date:** [May 25, 2021, 10:21am UTC](https://discuss.pytorch.org/t/get-current-lr-of-optimizer-with-adaptive-lr/24851/7 "2021-05-25T10:21:28Z")
This is important! Otherwise, it misleads. More info in this [pr](https://github.com/pytorch/pytorch/pull/26423) ---
**Author:** ![I-Love-U](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/i-love-u/32/8664_2.png) [@I-Love-U](https://discuss.pytorch.org/u/I-Love-U)\ **Post date:** [May 25, 2021, 1:42pm UTC](https://discuss.pytorch.org/t/get-current-lr-of-optimizer-with-adaptive-lr/24851/8 "2021-05-25T13:42:23Z")
Maybe you could add some additional methods for the optimizer instance: ```python # -*- coding: utf-8 -*- # @Time : 2020/12/19 # @Author : Lart Pang # @GitHub : https://github.com/lartpang import types from torch import nn from torch.optim import SGD, Adam, AdamW class OptimizerConstructor: def __init__ (self, model, initial_lr, mode, group_mode, cfg): """ A wrapper of the optimizer. :param model: nn.Module :param initial_lr: int :param mode: str :param group_mode: str :param cfg: A dict corresponding to your ``mode`` except the ``params`` and ``lr``. """ self.mode = mode self.initial_lr = initial_lr self.group_mode = group_mode self.cfg = cfg self.params = self.group_params(model) def construct_optimizer(self): if self.mode == "sgd": optimizer = SGD(params=self.params, lr=self.initial_lr, **self.cfg) elif self.mode == "adamw": optimizer = AdamW(params=self.params, lr=self.initial_lr, **self.cfg) elif self.mode == "adam": optimizer = Adam(params=self.params, lr=self.initial_lr, **self.cfg) else: raise NotImplementedError return optimizer def group_params(self, model): .... def __call__ (self): optimizer = self.construct_optimizer() optimizer.lr_groups = types.MethodType(get_lr_groups, optimizer) optimizer.lr_string = types.MethodType(get_lr_strings, optimizer) return optimizer def get_lr_groups(self): return [group["lr"] for group in self.param_groups] def get_lr_strings(self): return ",".join([f"{group['lr']:10.3e}" for group in self.param_groups]) ``` ---
**Author:** ![Prasad\_Raghavendra](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/prasad_raghavendra/32/24424_2.png) [@Prasad\_Raghavendra](https://discuss.pytorch.org/u/Prasad_Raghavendra)\ **Post date:** [March 22, 2025, 2:02am UTC](https://discuss.pytorch.org/t/get-current-lr-of-optimizer-with-adaptive-lr/24851/9 "2025-03-22T02:02:14Z")
I am assuming that works for `lr_scheduler`. What about optimiser?