I noticed that if I at some point I do for example sd = model.state_dict(), then the sd variable will be updated in every backward pass. How can I keep such variable from updating?
sd = model.state_dict()
I think it is a same question as below.
if you’ll update all parameters except a few parameter you don’t want to update, it is same that you don’t update a few parameters.
I think that is a different issue. I want to update all parameters. I just want to be able to keep in memory a copy of a previous state_dict.
I solved it saving the state_dict in disk and then loading it, but still I wonder how to do it in memory.
you can also do sd_clone = copy.deepcopy(sd) instead of writing it to disk and reading it again…
sd_clone = copy.deepcopy(sd)