How to modify weights of a layer before the layer is applied to input?

autograd
4

You can change the parameters, but would need to:

  • Make sure you are manipulating them inplace unless you explicitly want to replace the parameters with a new object. In this case, the optimizer will lose the reference to the old parameter and the new one won’t be updated unless you add it as a new param group.
  • Make sure to use the maipulation in a no_grad() context manager to skip Autograd tracking for this operation.
  • Make sure not to use the deprecated .data attribute.
1 Like