Doing operations on model parameters

I was wondering if there is any reliable method to manipulate model parameters, without any side-effects, so it could be deterministic in both GPU and CPU.
I want to be able add some numbers to a specific node of network or all of them, and even add or take average of two model from the same architecture.

Parameter manipulation can be performed inside a with torch.no_grad() block.
Iā€™m not sure to understand the requirement about deterministic results properly.
Do you want to make a GPU run match a CPU run or just GPU to GPU and CPU to CPU?

1 Like

Thank you for the answer.
Yes, I meant for both CPU and GPU, but for the manipulation of model parameters, I was looking for some pytorch method, that could manipulate the model parameters.
for others who are visiting this post:
A sample of manipulation from PySyft:

    params = model.named_parameters()
    dict_params = dict(params)
    with torch.no_grad():
        for name, param in dict_params.items():
            dict_params[name].set_(dict_params[name].data * scale)
1 Like