I want to use the same weights in the forward and backward steps in the training, but for the update step, i want to use different weights as old_weights in:
new_weights = old_weights - learning_rate * gradients
in fact, old_weights are different from the weights that are used in the forward and backward pass. How can i do this for model.parameters() in pytorch?
Thanks in advance