Reset a model's certain parameters using .data.clone() or .data.detach().clone()?

vision
1

Hello, may I consult about the difference in the effects of tensor.data.clone() and tensor.data.detach().clone()?

My goal is to interpolate between two parameters (of two models, respectively) and then assign the result to a new parameter (of a new model). My attempt is

new_param.data = 0.1 * old_param_1.data.clone() + 0.9 * old_param_2.data.clone()

I expect that new_param has no history of operations or gradients of the old parameters, and the modification of the new parameter or old parameters does not affect each other. Is my attempt correct?

Additionally, what if I do
new_param.data = 0.1 * old_param_1.data.detach().clone() + 0.9 * old_param_2.data.detach().clone()?

Is it necessary to add .detach()?

Thank you in advance!

2

Hi @JaneChen,

If you want the new_params to have no history of the previous operation you will need to detach those values in order to get that behavior.

Also, don’t call the .data attribute of a tensor as you’ll ignore some of the metadata of the tensor, you should call .detach().clone() directly on the tensor itself rather than the .data attribute.