I want to implement the following algorithm, from this book, section 13.6
Therefore, I need to be able to update all the network’s parameters by a constant factor.
I found the following code to acomplish that
state_dict = net.state_dict()
for name, param in state_dict.items():
# Transform the parameter as required.
transformed_param = param * 0.9
# Update the parameter.
state_dict[name].copy_(transformed_param)
but it doesn’t seem right, that all parameters have to pass through my code, rather than have some vector operation (which is implemented in a low level language, and is cuda supported).
I’m sure there is a better way of doing this.