Currently it seems that you’d input the LR from the start with optim.SGD
for example. How do you anneal it?
Or you could just recreate the optimizer
1 Like
If you recreate the optimizer, would it lose its weights, it seems to be? What do you mean by recreate?