How to adjust training hypter-parameters when training on more large dataset?

Hi, all
I have a question about training strategy optiming when training dataset get over large.
I am using AdamW optimizer, and ExponentialLR scheduler. we are training an Text-To-Speech model with 10 speakers and 40000 samples, in a single-card environment.
Initially we call scheduler.step() after each epoch, because the value of steps-per-epoch is not very big. that is fine.
Afterward, We should train another instance on a dataset of over 2000 speakers and millions of sampels, we found that the training became not so stable. It is more likely to fall into suboptimal result, and some times the training process will break down.
Because our GPU resouce can not afford a bigger batch_size, so this means that the value of steps-per-epoch is very big.
I think there are two ways to adjust the training hyper-parameters: 1、keep the mode that scheduler.step() after each epoch, but give a bigger lr_decay value of AdamW. 2、make scheduler.step() every N training steps instead of one-epoch.
I do not know which one is better, can anyone provide some experience? Thanks.