If I want to replace the model
model <- another_model
it’s easy:
model.load_state_dict(another_model.state_dict())
But how can I replace the model parameters according to a proportion?
model <- model * 0.99 + another_model * 0.01
If I want to replace the model
model <- another_model
it’s easy:
model.load_state_dict(another_model.state_dict())
But how can I replace the model parameters according to a proportion?
model <- model * 0.99 + another_model * 0.01
You could go through your state_dict
and set each value manually like this:
model.conv1.weights.data = 0.99 * model.conv1.weights.data + 0.01 * other_model.conv1.weights.data
Now I’ve found a nice solution!
for a, b in zip(model_a.parameters(), model_b.parameters()):
a.data = a.data * REPALCE_RATIO + b.data * (1 - REPALCE_RATIO)