Hi, after training my network I want to prune some of the parameters.
I get my weights and biases as ‘final_parameters = list(model.parameters())’ and append each element to a different list to play with them. However, when I need original parameters, list(model.parameters()) does not work because I noticed original parameters are also modified even though I modified them in a different list. So how can I handle this issue?
I think you have to clone parameters. Iterate over the list and clone them using
for param in model.parameters():
copied_model.append(param.clone())
Ok it works. But I don’t understand way original model parameters change when I modify them in a different list. It would be good to know.
Basically when you assign a new variable equal to another one, both points to the same memory array. If you modify one of both, both are modified.
When you clone it, you are indicating to compiler that you want to store the new variable in a new memory section, thus, both are independent after that.
Anyway it’s not a pytorch feature but python one. Some objects are mutable and other are inmutable.
You can read this is you are interested on (in ??¿ my english sucks) it.
python - Why does this assigned object share the same memory space as the original object? - Stack Overflow
Basically if you
>>> a=[0,1]
>>> b=a
>>> b
[0, 1]
>>> a
[0, 1]
>>> a[1]=155
>>> b
[0, 155]
>>> a
[0, 155]
As you can see, if you modify a, b, which points to the same memory adress, is also modfied.
I copy paste part of that link: credits to Blender
Python has mutable (e.g. lists, iterators, just about everything) and immutable objects (e.g. integers and strings). Assignment does not copy the object in either case. With immutable objects, all operations on them result in a new instance, so you won’t run into the problem of “modifying” an integer or a string like you do with mutable types.
Hope it helps
Regards
Juan
1 Like
I didn’t know that. Thank you so much.