lr=1e-4
If we set up the Differential lr for layer groups say
[{'params':group1,'lr': lr/10},{'param':group2,'lr':lr]
will this lr also varry as per Scheduler LR cycle ?
lr=1e-4
If we set up the Differential lr for layer groups say
[{'params':group1,'lr': lr/10},{'param':group2,'lr':lr]
will this lr also varry as per Scheduler LR cycle ?