I was wondering if there is any reliable method to manipulate model parameters, without any side-effects, so it could be deterministic in both GPU and CPU.
I want to be able add some numbers to a specific node of network or all of them, and even add or take average of two model from the same architecture.
Parameter manipulation can be performed inside a
with torch.no_grad() block.
I’m not sure to understand the requirement about deterministic results properly.
Do you want to make a GPU run match a CPU run or just GPU to GPU and CPU to CPU?
Thank you for the answer.
Yes, I meant for both CPU and GPU, but for the manipulation of model parameters, I was looking for some pytorch method, that could manipulate the model parameters.
for others who are visiting this post:
A sample of manipulation from PySyft:
params = model.named_parameters() dict_params = dict(params) with torch.no_grad(): for name, param in dict_params.items(): dict_params[name].set_(dict_params[name].data * scale)