Say I have a model, I can iterate through all the parameters using model.parameters(), but this is quite slow if I just want to do a consistent operation across all parameters, for example, adding a random noise to all parameters. If I have this vectorized representation of all the model parameters, I can just sample a random tensor and directly add to the representation.
You could pass the parameter list to torch._foreach_... methods, but note that these APIs are internal and their interface could change. Besides that you could also check e.g. this repository, which doesn’t seem to be maintained anymore but might still be helpful as a template in case you want to use a custom implementation.
Thank you very much, will look into that