Can i modify weights and gradients when training the network?

can i modify weights and gradients when training the network?
if yes, how to do it before and after updating parameters?

To modify gradients, use Variable hooks:

You can modify network weights directly. For example,, max=1e-3)

Thank you~ it’s helpful!

it seems that the hook will be activated during the backward process. And it will affect the gradient of the next layer in the backward automatically, right?

and i found a way to modify the weights and gradients too. But it’s not as awsome as yours, haha

in torch, i can modify weights and gradients directly by assign a tensor to it, like this = torch.ones(

and this has slight difference from the hook method if you use optim.step( ). But if you write you own step( ) method, and modify the gradients inside the scope of your step( ), this method and the hook method will do the same thing. Am I right?

And BTW, the weights are Tensor, but the gradients are Variable, very strange, haha. I know there must be some consideration like grad of grad? Anyway :smile:

Thank you for your help!

pytorch is awsome!